Image processing apparatus, image processing program and image processing method

ABSTRACT

To provide an image processing apparatus suitable for simplifying and accelerating a filtering process at the time of decoding an image and implementing an effective filter effect. A deblock filtering process judges whether or not difference values between values of subject pixels v 0  to v 7  and values of reference pixels r y0  to r y7  are exceeding a threshold, performs horizontal filter operation as to the subject pixels v 0  to v 7  based solely on the values of the subject pixels v 0  to v 7  and the values of the reference pixels r x0  and r x1 , determines whether or not to perform vertical filter operation as to the subject pixels v 0  to v 7  based solely on the judgment result, and performs the vertical filter operation as to the subject pixels v 0  to v 7  based solely on the values of the subject pixels v 0  to v 7  and the values of the reference pixels r y0  to r y7 .

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an apparatus, a program and amethod for, at the time of expanding an image by a block based oncompressed image data compressed by JPEG (Joint Photographic ExpertsGroup), MPEG (Moving Picture Experts Group) or the like, performing afiltering process to the expanded image, and in particular, to an imageprocessing apparatus, an image processing program and an imageprocessing method suitable for simplifying and accelerating thefiltering process at the time of decoding the image and implementing aneffective filter effect.

[0003] 2. Description of the Related Art

[0004] The JPEG method as a compression technology for a still pictureand the MPEG method as a compression technology for a motion picture arewidely known. These image compression methods handle a block of 8pixels×8 pixels as a unit of processing to perform a DCT (DiscreteCosine Transform) process. The DCT process is a process of decomposingan original image into spatial frequency components, and it can compressthe image by reducing spatially redundant information.

[0005] Incidentally, in case of adopting the JPEG method or the MPEGmethod, the following two types of noise are generated, which becomes acause of deteriorating image quality. To be more specific, as the JPEGmethod and the MPEG method handle the block of 8 pixels×8 pixels as aunit of processing, a boundary of the block may appear as the noisedepending on the image. This occurs because the DCT process is theprocess of eliminating an equivalent to a direct current component ofthe spatial frequency components of the image, and this process isperformed block by block so that an average of luminance of each blockbecomes different. This noise on the boundary portion is called blocknoise.

[0006] In addition, in the case of the DCT process by the JPEG methodand the MPEG method, a higher harmonic component of the spatialfrequency components of the image is eliminated as the redundantinformation so that there appears the noise in a portion of significantdifference in luminance from the surroundings. For instance, if the DCTprocess is performed in the case where there are significant variationsin luminance in the block such as a character existing in the backdropof a natural image, the higher harmonic component is eliminated fromdata constituting the original boundary portion and blurry noise appearsaround the boundary. This noise is called mosquito noise.

[0007] A technology for reducing the block noise and the mosquito noiseis described in Japanese Patent Laid-Open No. 3-46482. As for thetechnology described therein (hereafter, referred to as a first pastexample), a filtering process is performed in the case where a signallevel difference between boundary pixels located sandwiching theboundary between mutually adjacent blocks is larger than a firstthreshold, and of the boundary pixel and the pixels on a string in eachof the blocks, the signal level differences from the pixels adjacent tothat boundary pixel in the same block are smaller than a secondthreshold respectively.

[0008] At this time, in FIG. 14, the signal levels for pixel placementsX0 and X′0 are corrected based on difference values d0, d1 and d2 of thesignal levels S₁, S₀, S′₀ and S′₁ for the pixel placements X1 and X0 andthe pixel placements X′0 and X′1 located on both sides of the blockboundary. To be more specific, the new signal levels S_(0-new) andS′_(0-new) are as follows.

[0009] S_(0-new)=(S₁+2·S₀+S′₀)/4

[0010] S′_(0-new)=(S₀+2·S′₀+S′₁)/4

[0011] In addition, as another method of reducing the block noise andthe mosquito noise, a VM (Verification Model) for MPEG 4 and thetechnology described in Japanese Patent Laid-Open No. 11-98505(hereafter, referred to as a second past example) are also known. In thesecond past example, it is determined whether the vertical andhorizontal block boundaries shown in FIG. 15 are in a default mode or aDC offset mode and processed respectively.

[0012] In this case, pixel sets S0, S1 and S2 are defined with blockboundaries B1 and B2 as base points, and a deblocking mode isselectively determined as the default mode or the DC offset mode from amode determination value based on an extent of a block distortionphenomenon. And if it is determined as the default mode, frequencyinformation on the block boundary surroundings for each pixel isobtained by using a 4-point DCT kernel. If it is determined as the DCoffset mode in a mode determination stage, it is determined whether ornot execution of the DC offset mode is necessary, and if necessary, theblock distortion phenomenon is eliminated.

[0013] As for the first past example, however, it only corrects theboundary pixels so that there is a drawback of not being sufficient asthe process of eliminating the block distortion. In addition, as for theVM and the second past example, processing is complicated and thecomputation load is excessive.

[0014] In addition, in case of implementing these technologies byhardware, the processing is comparatively complicated, and so there is aproblem that it is difficult to render a circuit scale smaller. From theviewpoint of rendering it smaller, it is desirable to simplify theprocess as much as possible even if the image quality is sacrificed tosome extent. In addition, it is desirable, as there is a certain limitto a mountable memory capacity, to render the memory capacity to beoccupied for an operation in the filtering process as a possibleminimum.

SUMMARY OF THE INVENTION

[0015] Thus, the present invention has been achieved by paying attentionto these unsolved problems of the past technologies, and a first objectthereof is to provide an image processing apparatus, an image processingprogram and an image processing method suitable for simplifying andaccelerating the filtering process when decoding the image andimplementing an effective filter effect. In addition, a second objectthereof is to provide the image processing apparatus, the imageprocessing program and the image processing method suitable for reducingthe memory capacity to be occupied for the operation in the filteringprocess when decoding the image.

[0016] In order to attain the above objects, an image processingapparatus according to claim 1 of the present invention is an apparatusfor, based on compressed image data compressed by an image compressingprocess for performing a discrete cosine transform process and aquantization process by a predetermined block, performing a filteringprocess to an expanded image on or after expanding the image by theabove described block, wherein, in the case where a block adjacent to asubject block is a reference block, and furthermore, one pixel in theabove described subject block is a subject pixel, and one pixel in theabove described reference block is a reference pixel, the abovedescribed filtering process judges whether or not a difference valuebetween a value of the above described subject pixel and a value of theabove described reference pixel is exceeding a threshold, determineswhether or not to perform filter operation as to the above describedsubject pixel based solely on the above described judgement result, andperforms the filter operation as to the above described subject pixelbased solely on the value of the above described subject pixel and thevalue of the above described reference pixel.

[0017] In the case of such a configuration, it is judged by thefiltering process whether or not the difference value between the valueof the subject pixel and the value of the reference pixel is exceedingthe threshold on or after expanding the image by the block based on thecompressed image data so that it is determined whether or not to performthe filter operation as to the subject pixel based solely on thejudgment result. And if it is determined to perform the filteringprocess as to the subject pixel, the filter operation is performed basedsolely on the value of the subject pixel and the value of the referencepixel.

[0018] Here, to determine whether or not to perform the filter operationbased solely on one judgment result means to determine it based solelyon the result of judging whether or not the difference value between thevalue of the subject pixel and the value of the reference pixel isexceeding the threshold in case of using only one reference pixel forone subject pixel such as the case of only vertically performing thefiltering process to the subject pixel. In addition, in case of using aplurality of reference pixels for one subject pixel such as the case ofhorizontally and vertically performing the filtering process to thesubject pixel, the requirement of “only” is interpreted as to thefiltering process in each direction. To be more specific, in this case,a vertical filtering process meets the requirement of “only” in aconfiguration wherein it determines whether or not to perform a verticalfilter operation as to the subject pixel based solely on one judgmentresult but determines whether or not to perform a horizontal filteroperation as to the same subject pixel based on a plurality of judgmentresults. As a matter of course, a horizontal filtering process meets therequirement of “only” in the present invention in the opposite casethereto. Hereafter, it is the same as to the image processing apparatusaccording to claim 2, the image processing program according to claims 9and 10 and the image processing method according to claims 11 and 12.

[0019] In addition, to perform the filter operation based solely on twopixel values means to perform it based solely on the value of thesubject pixel and the value of the reference pixel in case of using onlyone reference pixel for one subject pixel such as the case of onlyvertically performing the filtering process to the subject pixel. Inaddition, in case of using a plurality of reference pixels for onesubject pixel such as the case of horizontally and vertically performingthe filtering process to the subject pixel, the requirement of “only” isinterpreted as to the filtering process in each direction. To be morespecific, in this case, the vertical filtering process meets therequirement of “only” in the configuration wherein it performs thevertical filter operation as to the subject pixel based solely on twopixel values but performs the horizontal filter operation as to the samesubject pixel based on three or more pixel values. As a matter ofcourse, the horizontal filtering process meets the requirement of “only”in the present invention in the opposite case thereto. Hereafter, it isthe same as to the image processing apparatus according to claim 2, theimage processing program according to claims 9 and 10 and the imageprocessing method according to claims 11 and 12.

[0020] Furthermore, the image processing apparatus according to claim 2of the present invention is the apparatus for, based on the compressedimage data compressed by the image compressing process for performingthe discrete cosine transform process and the quantization process bythe predetermined block, performing the filtering process to theexpanded image on or after expanding the image by the above describedblock, wherein, in the case where a block horizontally adjacent to thesubject block is a horizontal reference block, a block verticallyadjacent to the above described subject block is a vertical referenceblock, and furthermore, one pixel in the above described subject blockis a subject pixel, and one pixel in the above described horizontalreference block located on a boundary with the above described subjectblock and at the same vertical position as the above described subjectpixel is a horizontal reference pixel, and one pixel in the abovedescribed vertical reference block located on the boundary with theabove described subject block and at the same horizontal position as theabove described subject pixel is a vertical reference pixel, the abovedescribed filtering process judges whether or not a difference valuebetween a value of the above described subject pixel and a value of theabove described vertical reference pixel is exceeding a threshold, theabove described filtering process performs horizontal filter operationas to the above described subject pixel based solely on the value of theabove described subject pixel and the value of the above describedhorizontal reference pixel, the above described filtering processdetermines whether or not to perform vertical filter operation as to theabove described subject pixel based solely on the above describedjudgment result, and performs the vertical filter operation as to theabove described subject pixel based solely on the value of the abovedescribed subject pixel and the value of the above described verticalreference pixel.

[0021] In the case of such a configuration, the horizontal filteroperation as to the subject pixel is performed by the filtering processbased solely on the value of the subject pixel and the value of thehorizontal reference pixel on or after expanding the image by the blockbased on the compressed image data. In addition, it is judged whether ornot the difference value between the value of the subject pixel and thevalue of the vertical reference pixel is exceeding the threshold, and itis determined whether or not to perform the vertical filter operation asto the subject pixel based solely on the judgment result. And if it isdetermined to perform the vertical filter operation as to the subjectpixel, the vertical filter operation is performed based solely on thevalue of the subject pixel and the value of the vertical referencepixel.

[0022] Furthermore, the image processing apparatus according to claim 3of the present invention is characterized by, in the image processingapparatus according to claim 2, usably connecting to image storing meansfor storing the above described expanded image, and having storing meansfor work for storing the pixel data for performing the above describedfiltering process, and wherein, in the case where a plurality of pixelsin the above described subject block and horizontally successive fromthe boundary with the above described horizontal reference block are theabove described subject pixels, the above described filtering processcollectively reads the pixel data of a first pixel string comprised ofthe above described plurality of subject pixels and the above describedhorizontal reference pixel and the pixel data of a second pixel stringcomprised of vertical reference pixels corresponding to the abovedescribed respective subject pixels into the above described storingmeans for work, makes the above described judgment as to each of theabove described subject pixels based on the pixel data of the abovedescribed storing means for work, the above described filtering processperforms the horizontal filter operation as to the above describedrespective subject pixels based solely on the values of the abovedescribed subject pixels and the value of the above described horizontalreference pixel by referring to the pixel data of the above describedstoring means for work, the above described filtering process determineswhether or not to perform the vertical filter operation as to the abovedescribed respective subject pixels based solely on the above describedjudgment results corresponding to the subject pixels, and performs thevertical filter operation as to the above described respective subjectpixels based solely on the values of the above described subject pixelsand the values of the above described vertical reference pixels byreferring to the pixel data of the above described storing means forwork.

[0023] In the case of such a configuration, the pixel data of the firstpixel string and the pixel data of the second pixel string arecollectively read by the filtering process into the storing means forwork, and the horizontal filter operation as to the respective subjectpixels is performed by referring to the pixel data of the storing meansfor work and based solely on the values of the subject pixels and thevalue of the horizontal reference pixel. In addition, it is judgedwhether or not the difference value between the value of the abovedescribed subject pixel and the value of the above described verticalreference pixel is exceeding the threshold for each subject pixel basedon the pixel data of the storing means for work, and it is determinedwhether or not to perform the vertical filter operation as to eachsubject pixel based solely on the judgment result corresponding to thesubject pixel. And if it is determined to perform the vertical filteroperation as to the subject pixel, the vertical filter operation isperformed by referring to the pixel data of the storing means for workand based solely on the value of the subject pixel and the value of thevertical reference pixel.

[0024] Here, the storing means for work stores the pixel data by anymeans and in any period, and may have the pixel data stored in advanceor may have the pixel data stored by an input from the outside when theapparatus is in operation without having it stored in advance. It is thesame in case of storing the expanded image in the image storing means.

[0025] Furthermore, the image processing apparatus according to claim 4of the present invention is characterized by, in the image processingapparatus according to claim 3, constituting each of the above describedblocks as a rectangular image area comprised of 2n (n≧1) pieces of pixelhorizontally and 2m (m≧1) pieces of pixel vertically, and dividing itinto four areas of upper left, upper right, lower left and lower rightcomprised of n pieces of pixel horizontally and m pieces of pixelvertically, and further constituting the above described first pixelstring with the above described horizontal reference pixel and thepixels belonging to the same area of the above described four areas, theabove-described filtering process tries the above described filteroperation as to all the pixels of the above described expanded imagewhile repeating reading of the above described pixel data, the abovedescribed judgment and the above described filter operation as a seriesof processes, and as to the subject pixel belonging to the upper leftarea of the above described subject block, the above described filteringprocess uses the values of the reference pixels of the horizontalreference block and the vertical reference block adjacent to the abovedescribed subject block on the left of and above it for the abovedescribed judgment and the above described filter operation, as to thesubject pixel belonging to the upper right area of the above describedsubject block, the above described filtering process uses the values ofthe reference pixels of the horizontal reference block and the verticalreference block adjacent to the above described subject block on theright of and above it for the above described judgment and the abovedescribed filter operation, as to the subject pixel belonging to thelower left area of the above described subject block, the abovedescribed filtering process uses the values of the reference pixels ofthe horizontal reference block and the vertical reference block adjacentto the above described subject block on the left of and below it for theabove described judgment and the above described filter operation, andas to the subject pixel belonging to the lower right area of the abovedescribed subject block, the above described filtering process uses thevalues of the reference pixels of the horizontal reference block and thevertical reference block adjacent to the above described subject blockon the right of and below it for the above described judgment and theabove described filter operation.

[0026] In the case of such a configuration, the filtering process isperformed to all the pixels of the expanded image while the reading ofthe pixel data, the judgment and the filter operation are repeated as aseries of processes by the filtering process.

[0027] To be more specific, as for the subject pixels belonging to theupper left area of the subject block, the values of the reference pixelsof the horizontal reference block adjacent to the subject block on theleft of it and the values of the reference pixels of the verticalreference block adjacent to the subject block above it are used for thejudgment and the filter operation.

[0028] In addition, as for the subject pixels belonging to the upperright area of the subject block, the values of the reference pixels ofthe horizontal reference block adjacent to the subject block on theright of it and the values of the reference pixels of the verticalreference block adjacent to the subject block above it are used for thejudgment and the filter operation.

[0029] In addition, as for the subject pixels belonging to the lowerleft area of the subject block, the values of the reference pixels ofthe horizontal reference block adjacent to the subject block on the leftof it and the values of the reference pixels of the vertical referenceblock adjacent to the subject block below it are used for the judgmentand the filter operation.

[0030] In addition, as for the subject pixels belonging to the lowerright area of the subject block, the values of the reference pixels ofthe horizontal reference block adjacent to the subject block on theright of it and the values of the reference pixels of the verticalreference block adjacent to the subject block below it are used for thejudgment and the filter operation.

[0031] Here, while n needs to meet the condition of n≧1, n is n≧1 and isalso an integer since it does not take a decimal value due to the natureof the invention. In addition, while m needs to take a value which isindependent from n and meet the condition of m≧1, m is m≧1 and is alsoan integer since it does not take a decimal value due to the nature ofthe invention.

[0032] Furthermore, the image processing apparatus according to claim 5of the present invention is the image processing apparatus according toclaim 4, wherein the above described image storing means stores thepixel data of the pixels constituting the above described expanded imagein order of left to right and top to bottom, the above describedfiltering process refers to a pointer for the vertical reference pixelpointing at a storage address of the pixel data of the above describedsecond pixel string in the above described image storing means and readsthe pixel data of the above described second pixel string into the abovedescribed storing means for work, and furthermore, in the case where thenumber of horizontal pixels of the above described expanded image is Xand a vertical coordinate of the subject pixel in the above describedsubject block is y, as to each of the above described series ofprocesses, the above described filtering process adds a value equivalentto the number of the subject pixels to be collectively processed in theprocess to the value of the above described pointer for the verticalreference pixel, when the subject block to be processed next is aleftmost block in the above described expanded image and a remainder ondividing y by 2m is “0”, the above described filtering process subtracts2X from the value of the above described pointer for the verticalreference pixel, when the subject block to be processed next is theleftmost block in the above described expanded image and the remainderon dividing y by 2m is m, the above described filtering process adds 2mXto the value of the above described pointer for the vertical referencepixel, and when the subject block to be processed next is the leftmostblock in the above described expanded image and the remainder ondividing y by 2m is a value other than “0” or m, the above describedfiltering process subtracts X from the value of the above describedpointer for the vertical reference pixel.

[0033] In the case of such a configuration, in the filtering process,the pointer for the vertical reference pixel is referred to and thepixel data of the second pixel string is read into the storing means forwork. The pointer for the vertical reference pixel is updated on thefollowing condition.

[0034] First, as to each of the series of processes, the valueequivalent to the number of the subject pixels to be collectivelyprocessed in the process is added to the value of the pointer for thevertical reference pixel. Therefore, the pointer for the verticalreference pixel moves rightward in each of the series of processes bythe number of the subject pixels to be collectively processed in theprocess. As this move is just adding a predetermined number, it can beimplemented just by an adder in case of constituting it by the hardware.

[0035] Secondly, if the subject block to be processed next is theleftmost block in the expanded image and the remainder on dividing y by2m is a value other than “0” or m, X is subtracted from the value of thepointer for the vertical reference pixel. Therefore, when the series ofprocesses are completed as to all the pixels on the same horizontal linein the expanded image, the pointer for the vertical reference pixelmoves to the lowest step of the vertical reference block adjacent to thesubject block above it. As this move is just subtracting X, it can beimplemented just by the adder in case of constituting it by thehardware.

[0036] Thirdly, if the subject block to be processed next is theleftmost block in the expanded image and the remainder on dividing y by2m is m, 2mX is added to the value of the pointer for the verticalreference pixel. Therefore, when the series of processes are completedas to all the pixels on the same horizontal line in the expanded imageand the process moves on to the area in the lower half of the subjectblock, the pointer for the vertical reference pixel moves to the higheststep of the vertical reference block adjacent to the subject block belowit. As this move is just multiplying X by 2m and adding it, it can beimplemented just by a shifter and the adder in case of constituting itby the hardware.

[0037] Fourthly, if the subject block to be processed next is theleftmost block in the expanded image and the remainder on dividing y by2m is “0,” 2X is subtracted from the value of the pointer for thevertical reference pixel. Therefore, when the series of processes arecompleted as to all the pixels on the same horizontal line in theexpanded image and the process moves on to the block in the lower step,the pointer for the vertical reference pixel moves to the lowest step ofthe vertical reference block adjacent to the subject block above it. Asthis move is just multiplying X by 2 and subtracting it, it can beimplemented just by the shifter and the adder in case of constituting itby the hardware.

[0038] Furthermore, the image processing apparatus according to claim 6of the present invention is the image processing apparatus according toclaim 5, wherein, in the case where the pixel string comprised of a lefthalf of the pixels on the same horizontal line in the above describedsubject block is a first subject pixel string, the pixel stringcomprised of a right half thereof is a second subject pixel string, andfurthermore, the pixel string in the above described horizontalreference block and adjacent to the above described first subject pixelstring on the left of it and comprised of the same number of pixels asthat of the above described second subject pixel string is a firsthorizontal reference pixel string, and the pixel string in the abovedescribed horizontal reference block and adjacent to the above describedsecond subject pixel string on the right of it and comprised of the samenumber of pixels as that of the above described first subject pixelstring is a second horizontal reference pixel string, and furthermore,the pixel string comprised of the vertical reference pixelscorresponding to the subject pixels of the above described first subjectpixel string is a first vertical reference pixel string, and the pixelstring comprised of the vertical reference pixels corresponding to thesubject pixels of the above described second subject pixel string is asecond vertical reference pixel string, the above described filteringprocess reads the pixel data of the above described second subject pixelstring, the above described second horizontal reference pixel string,the above described first vertical reference pixel string and the abovedescribed second vertical reference pixel string into the abovedescribed storing means for work as to each of the above describedseries of processes, also reuses the pixel data of the above describedsecond subject pixel string and the above described second horizontalreference pixel string read in an immediately preceding process as thepixel data of the above described first horizontal reference pixelstring and the above described first subject pixel string, and performsthe above described judgment and the above described filter operation asto all the subject pixels of the above described first subject pixelstring and the above described second subject pixel string.

[0039] In the case of such a configuration, in the filtering process,the pixel data of the second subject pixel string, the second horizontalreference pixel string, the first vertical reference pixel string andthe second vertical reference pixel string is read into the storingmeans for work for each series of processes, and the pixel data of thesecond subject pixel string and the second horizontal reference pixelstring read in the immediately preceding process is also reutilized asthe pixel data of the first horizontal reference pixel string and thefirst subject pixel string. And the judgment and the filter operationare performed as to all the subject pixels of the first subject pixelstring and the second subject pixel string.

[0040] Here, either of the reading and reutilization may be performedfirst. However, in the case where a storage area for storing the pixeldata of the second subject pixel string and the second horizontalreference pixel string read in the immediately preceding process and thestorage area for storing the pixel data of the second subject pixelstring and the second horizontal reference pixel string to be read thistime are shared in the storing means for work, it is necessary toperform the reading after the reutilization.

[0041] In addition, the reutilization means to reutilize all or a partof the pixel data of the second subject pixel string and the secondhorizontal reference pixel string read in the immediately precedingprocess as all or a part of the pixel data of the first horizontalreference pixel string and the first subject pixel string respectively,and in case of utilizing a part thereof, it is necessary to include thedata of the pixel which can be the reference pixel in the pixel datautilized as the first horizontal reference pixel string.

[0042] In addition, the reutilization also means to generate (includingcopying) in the storing means for work the pixel data of the firsthorizontal reference pixel string and the first subject pixel stringbased on the pixel data of the second subject pixel string and thesecond horizontal reference pixel string read in the immediatelypreceding process, in addition to reutilizing in the storing means forwork the pixel data of the second subject pixel string and the secondhorizontal reference pixel string read in the immediately precedingprocess as-is as the pixel data of the first horizontal reference pixelstring and the first subject pixel string.

[0043] Furthermore, the image processing apparatus according to claim 7of the present invention is the image processing apparatus according toclaim 6, wherein the above described series of processes performed firstfor the above described expanded image reads the pixel data of the abovedescribed first subject pixel string, the above described second subjectpixel string, the above described second horizontal reference pixelstring, the above described first vertical reference pixel string andthe above described second vertical reference pixel string into theabove described storing means for work, and the above described seriesof processes performed for the second time and thereafter reads thepixel data of the above described second subject pixel string, the abovedescribed second horizontal reference pixel string, the above describedfirst vertical reference pixel string and the above described secondvertical reference pixel string into the above-described storing meansfor work, and also reuses the pixel data of the above described secondsubject pixel string and the above described second horizontal referencepixel string read in the immediately preceding process as the pixel dataof the above described first horizontal reference pixel string and theabove described first subject pixel string.

[0044] In the case of such a configuration, as for the expanded image,first in the series of processes, the pixel data of the first subjectpixel string, the second subject pixel string, the second horizontalreference pixel string, the first vertical reference pixel string andthe second vertical reference pixel string is read into the storingmeans for work.

[0045] As for the expanded image, in the series of processes performedfor the second time and thereafter, the pixel data of the second subjectpixel string, the second horizontal reference pixel string, the firstvertical reference pixel string and the second vertical reference pixelstring is read into the storing means for work, and the pixel data ofthe second subject pixel string and the second horizontal referencepixel string read in the immediately preceding process is alsoreutilized as the pixel data of the first horizontal reference pixelstring and the first subject pixel string respectively.

[0046] Therefore, while the pixel data of the five pixel strings is readonly at the start, the pixel data of the four pixel strings is read forthe second time and thereafter.

[0047] Furthermore, the image processing apparatus according to claims 8of the present invention is the image processing apparatus according toeither claim 6 or 7, wherein, in the case where the process of copyingthe pixel data of the above described second subject pixel string andthe above described second horizontal reference pixel string read in theimmediately preceding process as the pixel data of the above describedfirst horizontal reference pixel string and the above described firstsubject pixel string respectively is the first phase, the process ofreading the pixel data of the above described first vertical referencepixel string into the above described storing means for work is thesecond phase, the process of reading the pixel data of the abovedescribed second vertical reference pixel string into the abovedescribed storing means for work is the third phase, the process ofreading the pixel data of the above described second subject pixelstring into the above described storing means for work is the fourthphase, the process of reading the pixel data of the above describedsecond horizontal reference pixel string into the above describedstoring means for work is the fifth phase, the process of performing theabove described horizontal filter operation as to each subject pixel ofthe above described first subject pixel string based on the pixel dataof the above described first horizontal reference pixel string is thesixth phase, the process of performing the above described judgment andthe above described vertical filter operation as to each subject pixelof the above described first subject pixel string based on processingresults of the above described sixth phase and the pixel data of theabove described first vertical reference pixel string is the seventhphase, the process of performing the above described horizontal filteroperation as to each subject pixel of the above described second subjectpixel string based on the pixel data of the above described secondhorizontal reference pixel string is the eighth phase, and the processof performing the above described judgment and the above describedvertical filter operation as to each subject pixel of the abovedescribed second subject pixel string based on the processing results ofthe above described eighth phase and the pixel data of the abovedescribed second vertical reference pixel string is the ninth phase, theabove described series of processes performs the above described firstphase, the above described second phase, the above described thirdphase, the above described fourth phase and the above described fifthphase in that order, performs the above described sixth phase and theabove described seventh phase in that order, performs the abovedescribed fifth phase, and the above described eighth phase and theabove described ninth phase in that order, and also starts performingthe above described second phase and the above described sixth phase atthe same time and performing the above described third phase and theabove described seventh phase at the same time.

[0048] In the case of such a configuration, as the first phase isperformed by the series of processes, the pixel data of the secondsubject pixel string and the second horizontal reference pixel stringread in the immediately preceding process is copied as the pixel data ofthe first horizontal reference pixel string and the first subject pixelstring.

[0049] Although the pixel data of the first horizontal reference pixelstring and the first subject pixel string is necessary to perform thehorizontal filter operation as to each subject pixel of the firstsubject pixel string, it is available by performing the first phase, sothat the sixth phase can be performed when starting to perform thesecond phase. Therefore, performance of the second phase and the sixthphase is simultaneously started. Thus, the pixel data of the firstvertical reference pixel string is read into the storing means for work,and the horizontal filter operation is performed as to each subjectpixel of the first subject pixel string based on the pixel data of thefirst horizontal reference pixel string.

[0050] Furthermore, although the results of the horizontal filteroperation and the pixel data of the first vertical reference pixelstring are necessary in order to perform the judgment and the verticalfilter operation as to each subject pixel of the first subject pixelstring, they are available by performing the second and sixth phases, sothat the seventh phase can be performed when starting to perform thethird phase. Therefore, performance of the third and seventh phases aresimultaneously started. Thus, the pixel data of the second verticalreference pixel string is read into the storing means for work, and thejudgment and vertical filter operation are performed as to each subjectpixel of the first subject pixel string based on the processing resultsof the sixth phase and the pixel data of the first vertical referencepixel string.

[0051] And the fourth and fifth phases are performed in that order sothat the pixel data of the second subject pixel string and the secondhorizontal reference pixel string is read into the storing means forwork.

[0052] Although the pixel data of the second horizontal reference pixelstring and the second subject pixel string is necessary in order toperform the horizontal filter operation as to each subject pixel of thesecond subject pixel string, it is available by performing the fourthand fifth phases, so that the eighth phase can be performed followingthe end of the fifth phase. Thus, the horizontal filter operation isperformed as to each subject pixel of the second subject pixel stringbased on the pixel data of the second horizontal reference pixel string.

[0053] Furthermore, although the results of the horizontal filteroperation and the pixel data of the second vertical reference pixelstring are necessary in order to perform the judgment and the verticalfilter operation as to each subject pixel of the second subject pixelstring, they are available by performing the third and eighth phases, sothat the ninth phase can be performed following the end of the eighthphase. Thus, the judgment and vertical filter operation are performed asto each subject pixel of the second subject pixel string based on theprocessing results of the eighth phase and the pixel data of the secondvertical reference pixel string.

[0054] On the other hand, in order to attain the above objects, theimage processing program according to claim 9 of the present inventionis a program for, based on the compressed image data compressed by theimage compressing process for performing the discrete cosine transformprocess and the quantization process by the predetermined block, causingthe computer to perform the filtering process to the expanded image onor after expanding the image by the above described block, wherein, inthe case where a block adjacent to the subject block is the referenceblock, and furthermore, one pixel in the above described subject blockis the subject pixel, and one pixel in the above described referenceblock is the reference pixel, the above described filtering processjudges whether or not the difference value between the value of theabove described subject pixel and the value of the above describedreference pixel is exceeding the threshold, determines whether or not toperform the filter operation as to the above described subject pixelbased solely on the above described judgment result, and performs thefilter operation as to the above described subject pixel based solely onthe value of the above described subject pixel and the value of theabove described reference pixel.

[0055] In the case of such a configuration, the action equivalent tothat of the image processing apparatus according to claim 1 can beobtained by having the program read by the computer and having theprocess executed by the computer according to the read program.

[0056] Furthermore, the image processing program according to claim 10of the present invention is the program for, based on the compressedimage data compressed by the image compressing process for performingthe discrete cosine transform process and the quantization process bythe predetermined block, causing a computer to perform the filteringprocess to the expanded image on or after expanding the image by theabove described block, wherein, in the case where the block horizontallyadjacent to the subject block is the horizontal reference block, theblock vertically adjacent to the above described subject block is thevertical reference block, and furthermore, one pixel in the abovedescribed subject block is the subject pixel, and one pixel in the abovedescribed horizontal reference block located on the boundary with theabove described subject block and at the same vertical position as theabove described subject pixel is the horizontal reference pixel, and onepixel in the above described vertical reference block located on theboundary with the above described subject block and at the samehorizontal position as the above described subject pixel is a verticalreference pixel, the above described filtering process judges whether ornot the difference value between the value of the above describedsubject pixel and the value of the above described vertical referencepixel is exceeding the threshold, the above described filtering processperforms the horizontal filter operation as to the above describedsubject pixel based solely on the value of the above described subjectpixel and the value of the above described horizontal reference pixel,the above described filtering process determines whether or not toperform the vertical filter operation as to the above described subjectpixel based solely on the above described judgment result, and performsthe vertical filter operation as to the above described subject pixelbased solely on the value of the above described subject pixel and thevalue of the above described vertical reference pixel.

[0057] In the case of such a configuration, the action equivalent tothat of the image processing apparatus according to claim 2 can beobtained by having the program read by the computer and having theprocess executed by the computer according to the read program.

[0058] On the other hand, in order to attain the above objects, theimage processing method according to claim 11 of the present inventionis the method for, based on the compressed image data compressed by theimage compressing process for performing the discrete cosine transformprocess and the quantization process by the predetermined block,performing the filtering process to the expanded image on or afterexpanding the image by the above described block, wherein, in the casewhere the block adjacent to the subject block is the reference block,and furthermore, one pixel in the above described subject block is thesubject pixel, and one pixel in the above described reference block isthe reference pixel, the above described filtering process judgeswhether or not the difference value between the value of the abovedescribed subject pixel and the value of the above described referencepixel is exceeding the threshold, determines whether or not to performthe filter operation as to the above described subject pixel basedsolely on the above described judgment result, and performs the filteroperation as to the above described subject pixel based solely on thevalue of the above described subject pixel and the value of the abovedescribed reference pixel.

[0059] Furthermore, the image processing method according to claim 12 ofthe present invention is the method for, based on the compressed imagedata compressed by the image compressing process for performing thediscrete cosine transform process and the quantization process by thepredetermined block, performing the filtering process to the expandedimage on or after expanding the image by the above described block,wherein, in the case where the block horizontally adjacent to thesubject block is the horizontal reference block, the block verticallyadjacent to the above described subject block is the vertical referenceblock, and furthermore, one pixel in the above described subject blockis the subject pixel, and one pixel in the above described horizontalreference block located on the boundary with the above described subjectblock and at the same vertical position as the above described subjectpixel is a horizontal reference pixel, and one pixel in the abovedescribed vertical reference block located on the boundary with theabove described subject block and at the same horizontal position as theabove described subject pixel is the vertical reference pixel, the abovedescribed filtering process judges whether or not the difference valuebetween the value of the above described subject pixel and the value ofthe above described vertical reference pixel is exceeding the threshold,the above described filtering process performs the horizontal filteroperation as to the above described subject pixel based solely on thevalue of the above described subject pixel and the value of the abovedescribed horizontal reference pixel, the above described filteringprocess determines whether or not to perform the vertical filteroperation as to the above-described subject pixel based solely on theabove described judgment result, and performs the vertical filteroperation as to the above described subject pixel based solely on thevalue of the above described subject pixel and the value of the abovedescribed vertical reference pixel.

BRIEF DESCRIPTION OF THE DRAWINGS

[0060]FIG. 1 is a block diagram showing a configuration of a computersystem to which the present invention is applied to;

[0061]FIG. 2 is a diagram showing a subject block which is a subject ofa deblock filtering process in a reproduced image and a block adjacentthereto;

[0062]FIG. 3 is a bar graph showing values of subject pixels v₀ to v₇ onthe same horizontal line in mutually adjacent blocks A and B;

[0063]FIG. 4 is a diagram showing the subject block which is the subjectof the deblock filtering process in the reproduced image and the blockadjacent thereto;

[0064]FIG. 5 is a flowchart showing a main control process;

[0065]FIG. 6 is a flowchart showing a pixel data reading process;

[0066]FIG. 7 is a flowchart showing a first horizontal filteringprocess;

[0067]FIG. 8 is a flowchart showing a first vertical filtering process;

[0068]FIG. 9 is a flowchart showing a second horizontal filteringprocess;

[0069]FIG. 10 is a flowchart showing a second vertical filteringprocess;

[0070]FIG. 11 is a flowchart showing a pointer updating process;

[0071]FIG. 12 is a time chart showing a process of performing thedeblock filtering process;

[0072]FIG. 13 is a diagram for explaining the case of copying andreutilizing pixel data;

[0073]FIG. 14 is a diagram showing a filtering process by an imageprocessing system in the past; and

[0074]FIG. 15 is a diagram showing pixels referred to by the imageprocessing system in the past.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0075] Hereafter, the embodiments of the present invention will bedescribed by referring to the drawings. FIGS. 1 to 13 are diagramsshowing the embodiments of an image processing apparatus, an imageprocessing program and an image processing method related to the presentinvention.

[0076] These embodiments apply the image processing apparatus, imageprocessing program and image processing method related to the presentinvention to the case of, when decoding motion picture data compressedin an MPEG format and reproducing a motion picture by a multitasking OS(operating system) in a computer 100 as shown in FIG. 1, performing afiltering process for reducing block noise to a reproduced image(hereafter, referred to as a deblock filtering process).

[0077] First, a configuration of a computer system to which the presentinvention is applied will be described by referring to FIG. 1. FIG. 1 isa block diagram showing the configuration of the computer system towhich the present invention is applied.

[0078] As shown in FIG. 1, the computer 100 is comprised of a CPU 30 forcontrolling calculation and the entire system based on a controlprogram, an ROM 32 for having the control program of the CPU 30 and soon stored in a predetermined area in advance, an RAM 34 for storing dataread from the ROM 32 and calculation results necessary in a calculationprocess of the CPU 30, an LCDC (Liquid Crystal Display Controller) 36for converting the data of a specific area of the RAM 34 into an imagesignal and outputting it to an LCD 44, a decoder 38 for decoding motionpicture data, a post filter portion 40 for reading the data of thespecific area of the RAM 34 and performing the deblock filteringprocess, an RGB converting portion 42 for performing coloring conversionto the data from the post filter portion 40, and an LCD 44 forperforming display based on the data from the RGB converting portion 42.And the CPU 30, ROM 32, RAM 34, LCDC 36 and decoder 38 are connectedmutually and to be able to send and receive the data by a bus 39 whichis a signal line for transferring the data.

[0079] The RAM 34 has a VRAM 35, as a specific area, for storing thedata to be displayed on the LCD 44, and the VRAM 35 is capable of beingaccessed independently by the apparatus connected to the bus 39 such asthe CPU 30 and by the post filter portion 40. In addition, the VRAM 35has at least two areas capable of storing image data of one screen ofthe LCD 44 so that one of the areas is for expansion of the motionpicture data by the decoder 38 and the other area is for the reading bythe post filter portion 40, and these areas are alternately switched andused in a predetermined cycle.

[0080] The LCDC 36 controls the post filter portion 40 and the RGBconverting portion 42 and thereby reads the data of the VRAM 35sequentially from a first address in a predetermined cycle so as tooutput the read data to the LCD 44 by converting it into the imagesignal. To be more specific, under the control of the LCDC 36, the postfilter portion 40 reads the data from the VRAM 35, performs the deblockfiltering process to the read data and outputs it to the RGB convertingportion 42 so that, under the control of the LCDC 36, the RGB convertingportion 42 performs RGB conversion to the data from the post filterportion 40 and outputs it to the LCD 44.

[0081] The decoder 38 sequentially decodes the image data included inthe motion picture data every predetermined time T (20 [ms] forinstance). To be more specific, on an instruction to start decoding fromthe CPU 30, the motion picture data compressed in the MPEG format isread from the RAM 34, and the image data is decoded in the MPEG formatto the VRAM 35 based on the read motion picture data. The VRAM 35 hasthe image data expanded therein in the units of pixel data (YUV data)showing a degree of color of one pixel of the LCD 44. In addition, thedecoder 38 performs an IDCT (Inverse Discrete Cosine Transform) processand a motion compensation process of the decoding process in the MPEGformat. Moreover, a compression process and a decoding process in theMPEG format are per past examples.

[0082] Next, the configuration of the post filter portion 40 will bedescribed in detail.

[0083] The post filter portion 40 has the configuration and internalprocessing thereof implemented by hardware, has a buffer capable ofstoring the pixel data of 21 pixels, divides the reproduced image into aplurality of blocks, and while reading the pixel data from the VRAM 35to the buffer, it performs the deblock filtering process in the unit ofa block based on the read pixel data. Each block is constituted as arectangular image area comprised of 8 pieces of pixel horizontally and 8pieces of pixel vertically, and furthermore, is divided into four areasof upper left, upper right, lower left and lower right comprised of 4pieces of pixel horizontally and 4 pieces of pixel vertically.

[0084] A principle of the deblock filtering process related to thepresent invention will be described in detail by referring to FIG. 2 toFIG. 4. FIG. 2 and FIG. 4 are diagrams showing the subject block whichis the subject of the deblock filtering process and the block adjacentthereto.

[0085] In FIG. 2, on the left and on the right of a subject block 10, ahorizontal reference block 12 is adjacent thereto sandwiching a boundaryBL0 and a horizontal reference block 14 is adjacent thereto sandwichinga boundary BL1 respectively. Furthermore, on the upper and the lower ofthe subject block 10, a vertical reference block 16 is adjacent theretosandwiching a boundary BL2 and a vertical reference block 18 is adjacentthereto sandwiching a boundary BL3 respectively.

[0086] In the deblock filtering process related to the presentinvention, the pixels on the same horizontal line in the subject block10 are the subject pixels, and the subject 8 pixels are collectivelyprocessed by filtering. In the example of FIG. 2, the pixels on the samehorizontal line belonging to an upper half of the subject block 10 aresubject pixels v₀ to v₇. Here, in the case where the pixel stringcomprised of the four pixels of the left half of the pixels on the samehorizontal line is a first subject pixel string L_(SL), and the pixelstring comprised of the four pixels of the right half is a secondsubject pixel string L_(SR), the first subject pixel string L_(SL)includes the subject pixels v₀ to v₃ in order from the left, and thesecond subject pixel string L_(SR) includes the subject pixels v₄ to v₇in order from the left.

[0087] In addition, in the case where the pixel string in the horizontalreference block 12 adjacent to the first subject pixel string L_(SL) onthe left of it and comprised of four pixels is a first horizontalreference pixel string L_(XL) and the pixel string in the horizontalreference block 14 adjacent to the second subject pixel string L_(SR) onthe right of it and comprised of four pixels is a second horizontalreference pixel string L_(XR), the first horizontal reference pixelstring L_(XL) includes a reference pixel r_(x0) at its right end, andthe second horizontal reference pixel string L_(XR) includes a referencepixel r_(x1) at its left end. The reference pixel r_(x0) is used forhorizontal filter operation as to the subject pixels v₀ to v₃, and thereference pixel r_(x1) is used for the horizontal filter operation as tothe subject pixels v₄ to v₇.

[0088] In addition, in the case where the pixel string of the loweststep in the vertical reference block 16 located above the first subjectpixel string L_(SL) and comprised of four pixels is a first verticalreference pixel string L_(YL) and the pixel string of the lowest step inthe vertical reference block 16 located above the second subject pixelstring L_(SR) and comprised of four pixels is a second verticalreference pixel string L_(YR), the first vertical reference pixel stringL_(YL) includes the reference pixels r_(y0) to r_(y3) in order from theleft, and the second vertical reference pixel string L_(YR) includes thesubject pixels r_(y4) to r_(y7) in order from the left. The referencepixels r_(y0) to r_(y3) are used for vertical filter operation as to thesubject pixels v₀ to v₃, and the reference pixel r_(y4) to r_(y7) areused for the vertical filter operation as to the subject pixels v₄ tov₇.

[0089] In the example of FIG. 2, in case of performing the horizontalfilter operation as to the subject pixels v₀ to v₇ by the deblockfiltering process related to the present invention, the values of thesubject pixels v_(xo)′ to v_(x7)′ after the deblock filtering processare calculated by the following equations (1) to (8) for the filteroperation. Moreover, v₀ to v₇ indicate the values of the subject pixels,and r_(x0) to r_(x7) indicate the values of the reference pixels in thefollowing equations (1) to (8).

v _(x0)′=(v ₀ +r _(x0))/2+(v ₀ −r _(x0))/8  (1)

v _(x1)′=(v ₁ +r _(x0))/2+(v ₁ −r _(x0))/4  (2)

v _(x2) ′=v ₂−(v ₂ −r _(x0))/8  (3)

v _(x3) ′=v ₈  (4)

v _(x4) ′=v ₄  (5)

v _(x5)′=(v ₅ +r _(x1))/2+(v ₅ −r _(x1))/4  (6)

v _(x6)′=(v ₆ +r _(x1))/2+(v ₆ −r _(x1))/8  (7)

v _(x7)′=(v ₇ +r _(x1))/2  (8)

[0090] The above equations (1) to (8) for the filter operation arederived as follows. FIG. 3 is a bar graph showing the values of thesubject pixels v₀ to v₇ on the same horizontal line in the mutuallyadjacent blocks A and B.

[0091] As shown in FIG. 3A, in the mutually adjacent blocks A and B, anequivalent to a direct current component of the spatial frequencycomponent of the image is eliminated by a DCT process on compression, sothat differences in levels occur due to different averages of luminance.The differences in levels become block noise. Thus, the case of smoothlyconnecting the images of the blocks A and B is considered in order toreduce the block noise. An average V_(A) of the four pixels in the blockA and an average V_(B) of the four pixels in the block B can becalculated by the following equations (9) and (10).

V _(A)=(v ₄ +v ₅ +v ₆ +v ₇)/4  (9)

V _(B)=(v ₀ +v ₁ +v ₂ +v ₃)/4  (10)

[0092] If the amount of departure from the averages V_(A) and V_(B) ofeach subject pixel value vi is δv_(i), each subject pixel value v_(i)can be represented by the following equations (11) and (12).

v _(i) =V _(A) +δv _(i)(i=4 to 7)  (11)

v _(i) =V _(B) +δv _(i)(i=0 to 3)  (12)

[0093] Now, in order to smoothly connect the images of the blocks A andB, the difference between the averages V_(A) and V_(B) should be “0,”and each subject pixel value v_(i) should be corrected stepwise as shownin FIG. 3B. As the difference in level Δ of the steps is as follows.

Δ=(V _(B) −V _(A))/8  (13)

[0094] the corrected value v_(x0)′ of the subject pixel value v₀ can becalculated by the following equation (14) for instance. $\begin{matrix}\begin{matrix}{v_{x0}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} + \Delta + {\delta \quad v_{0}}}} \\{= {{\left( {{5V_{B}} + {3V_{A}}} \right)/8} + {\delta \quad v_{0}}}} \\{= {{\left\{ {{5\left( {v_{0} - {\delta \quad v_{0}}} \right)} + {3\left( {v_{7} - {\delta \quad v_{7}}} \right)}} \right\}/8} + {\delta \quad v_{0}}}} \\{= {\left\{ {{5v_{0}} + {3v_{7}} + {3\left( {{\delta \quad v_{0}} - {\delta \quad v_{7}}} \right)}} \right\}/8}}\end{matrix} & (14)\end{matrix}$

[0095] Here, while the above equation (14) must be observed in order tocalculate a correct value, (δv₀−δv₇) is assumed to be small and isomitted in the above equation (14) so as to obtain an approximateexpression which is the following equation (15).

v _(x0)′=(5v ₀+3v ₇)/8  (15)

[0096] In addition, as for the other subject pixels in the block B, thecorrected values v_(x1)′ to v_(x3)′ can be calculated likewise by thefollowing equations (16) to (18) $\begin{matrix}\begin{matrix}{v_{x1}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} + {2\quad \Delta} + {\delta \quad v_{1}}}} \\{= {\left\{ {{6v_{1}} + {2v_{7}} + {2\left( {{\delta \quad v_{1}} - {\delta \quad v_{7}}} \right)}} \right\}/8}}\end{matrix} & (16) \\\begin{matrix}{v_{x2}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} + {3\quad \Delta} + {\delta \quad v_{2}}}} \\{= {\left\{ {{7v_{2}} + v_{7} + \left( {{\delta \quad v_{2}} - {\delta \quad v_{7}}} \right)} \right\}/8}}\end{matrix} & (17) \\\begin{matrix}{v_{x3}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} + {4\quad \Delta} + {\delta \quad v_{3}}}} \\{= v_{3}}\end{matrix} & (18)\end{matrix}$

[0097] Thus, the approximate expressions of the following equations (19)and (20) are obtained.

v _(x1)′=(3v ₁ +v ₇)/4  (19)

v _(x2)′=(7v ₂ +v ₇)/8  (20)

[0098] In addition, as for the subject pixels in the block A, they arethe same except the difference that the value of the reference pixel isv₀, and the corrected values v_(x4)′ to v_(x7)′ can be calculated by thefollowing equations (21) to (24). $\begin{matrix}\begin{matrix}{v_{x4}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} - {3\Delta} + {\delta \quad v_{4}}}} \\{= {\left\{ {{7v_{4}} + v_{0} + \left( {{\delta \quad v_{4}} - {\delta \quad v_{0}}} \right)} \right\}/8}}\end{matrix} & (21) \\\begin{matrix}{v_{x5}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} - {2\quad \Delta} + {\delta \quad v_{5}}}} \\{= {\left\{ {{6v_{5}} + {2v_{0}} + {2\left( {{\delta \quad v_{5}} - {\delta \quad v_{0}}} \right)}} \right\}/8}}\end{matrix} & (22) \\\begin{matrix}{v_{x6}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} - \Delta + {\delta \quad v_{6}}}} \\{= {\left\{ {{5v_{6}} + {3v_{0}} + \left( {{\delta \quad v_{6}} - {\delta \quad v_{0}}} \right)} \right\}/8}}\end{matrix} & (23) \\\begin{matrix}{v_{x7}^{\prime} = {{\left( {V_{B} + V_{A}} \right)/2} + {\delta \quad v_{7}}}} \\{= {\left\{ {v_{7} + v_{0} + {2\delta \quad v_{7}}} \right\}/2}}\end{matrix} & (24)\end{matrix}$

[0099] Thus, the approximate expressions of the following equations (25)and (28) are obtained.

v _(x4) ′=v ₄  (25)

v _(x5)′=(3v ₅ +v ₀)/4  (26)

v _(x6)′=(5v ₆+3v ₀)/8  (27)

v _(x7)′=(v ₇ +v ₀)/2  (28)

[0100] In addition, as bit shift rather than division can accelerate theprocessing in actual calculation, the above equations (15), (19), (20),(18), (25) to (28) are transformed to the above equations (1) to (8) andused. This deblock filtering process is performed to a horizontal pixelstring and a vertical pixel string respectively. As for the amount ofcalculation, it requires addition and subtraction twice respectively andbit shift four times for one pixel. The bit shift does not become muchof a load when rendering it as hardware. As a post filter in the pasttook a weighting average of maximum 7×7 pixels per pixel, the-amount ofcalculation can be significantly reduced compared thereto.

[0101] Likewise, as for the example in FIG. 2, in case of performingvertical filter operation as to the subject pixels v_(x0)′ to v_(x7)′ bythe deblock filtering process related to the present invention, thevalues v_(yi)′ (I=0 to 7) of the subject pixels after the deblockfiltering process can be calculated by a filter operation equation ofthe following equation (29). Moreover, r_(y1) indicates the value of thereference pixel in the following equation (29) to (36).

v _(yi) ′=v _(xi)′−(v _(xi) ′−r _(yi))/8  (29)

[0102] As for the example in FIG. 2, the subject pixels v₀ to v₇ arepositioned in the third step in the subject block 10, the filteroperation equation of the above equation (29) is corresponding to theabove equation (3). Therefore, when the subject pixels v₀ to v₇ arepositioned in the first, second and fourth steps, the filter operationequations are corresponding to the above equations (1), (2) and (4) asshown in the following equation (30) to (32), and when the subjectpixels v₀ to V₇ are positioned in the fifth to eighth steps, the filteroperation equations are corresponding to the above equations (5) to (8)as shown in the following equation (33) to (36). However, when thesubject pixels v₀ to v₇ are positioned in the fifth to eighth steps, asshown in FIG. 4, the first vertical reference pixel string L_(YL) is thepixel string of the highest step in the vertical reference block 18 andis located below the first subject pixel string L_(SL) and comprised offour pixels, and the second vertical reference pixel string L_(YR) isthe pixel string of the highest step in the vertical reference block 18and is located below the second subject pixel string L_(SR) andcomprised of four pixels.

v _(yi)′=(v _(xi) ′+r _(yi))/2+(v _(xi) ′−r _(yi))/8  (30)

v _(yi)′=(v _(xi) ′+r _(yi))/2+(v _(xi) ′−r _(yi))/4  (31)

v _(yi) ′=v _(xi)′  (32)

v _(yi) ′=v _(xi)′  (33)

v _(yi)′=(v _(xi) ′+r _(yi))/2+(v _(xi) ′−r _(yi))/4  (34)

v _(yi)′=(v _(xi) ′+r _(yi))/2+(v _(xi) ′−r _(yi))/8  (35)

v _(yi)′=(v _(xi) ′+r _(yi))/2  (36)

[0103] To be more specific, the deblock filtering process related to thepresent invention can be implemented by a main control process, a pixeldata reading process, a first horizontal filter processing, a firstvertical filter processing, a second horizontal filter processing and asecond vertical filter processing shown in the flowcharts in FIGS. 5 to10.

[0104] First, the main control process will be described in detail byreferring to FIG. 5. FIG. 5 is a flowchart showing the main controlprocess.

[0105] If performed in the post filter portion 40, the main controlprocess moves on to a step S100 as shown in FIG. 5.

[0106] In the step S100, it sets as a pointer for the subject pixel *sxan address at which the pixel data of the leftmost pixel of the firstsubject pixel string L_(SL) to be processed first in the VRAM 35 isstored, and sets as a pointer for the vertical reference pixel *cy anaddress at which the pixel data of the left most pixel of the firstvertical reference pixel string L_(YL) to be processed first in the VRAM35 is stored respectively. Here, the pointer for the subject pixel *sxis used to read from the VRAM 35 the pixel data of the first subjectpixel string L_(SL), the second subject pixel string L_(SR) and thesecond horizontal reference pixel string L_(XR). In addition, thepointer for the vertical reference pixel *cy is used to read from theVRAM 35 the first vertical reference pixel string L_(YL) and the secondvertical reference pixel string L_(YR).

[0107] Next, it moves onto a step S102, and refers to the pointer forthe subject pixel *sx and the pointer for the vertical reference pixel*cy, and outputs a read request to read the first subject pixel stringL_(SL), second subject pixel string L_(SR), second horizontal referencepixel string L_(XR), first vertical reference pixel string L_(YL) andsecond vertical reference pixel string L_(YR) from the VRAM 35, andmoves on to a step S104.

[0108] In the step S104, it judges, by a first vertical filteringprocess and a second vertical filtering process mentioned later, whetheror not all the pixel data after the filtering process has been outputtedto the RGB converting portion 42 as to the subject pixels of the firstsubject pixel string L_(SL) and second subject pixel string L_(SR), andif it judges that all the pixel data after the filtering process hasbeen outputted thereto (Yes), it moves on to a step S106, but if itjudges otherwise (No), it waits in the step S104 until all the pixeldata after the filtering process is outputted to the RGB convertingportion 42.

[0109] In the step S106, it performs a pointer updating process ofupdating the pointer for the subject pixel *sx and the pointer for thevertical reference pixel *cy, and moves on to a step S108 to judgewhether or not the filtering process has been finished as to all thepixels of the reproduced image, and if it judges that the filteringprocess has been finished as to all the pixels of the reproduced image(Yes), it finishes the series of processes and has the original processreturned.

[0110] On the other hand, if it judges that the filtering process hasnot been finished as to all the pixels of the reproduced image (No), itmoves on to the step S102.

[0111] Next, the pixel data reading process will be described in detailby referring to FIG. 6. FIG. 6 is a flowchart showing the pixel datareading process.

[0112] The pixel data reading process is the process of, according tothe read request, reading the first subject pixel string L_(SL), secondsubject pixel string L_(SR), second horizontal reference pixel stringL_(XR), first vertical reference pixel string L_(YL) and second verticalreference pixel string L_(YR) from the VRAM 35 to the buffer, and ifperformed in the post filter portion 40, it first moves on to a stepS200 as shown in FIG. 6.

[0113] In the step S200, it judges whether or not the read request hasbeen outputted, and if it judges the read request has been outputted(Yes), it moves on to a step S202, but if it judges otherwise (No), itwaits in the step S200 until the read request is outputted.

[0114] In the step S202, it refers to the pointer for the subject pixel*sx and reads U data showing a U component of the pixel data of thesubject pixels v₀ to v₃ of the first subject pixel string L_(SL). Asthis embodiment constitutes the U data as one piece of data for everytwo pixels, it reads the U data of the subject pixels v₀ and v₁ and theU data of the subject pixels v₂ and v₃ in the step S202. Moreover, asfor the U data, priority is given to acceleration of processing and nodeblock filtering process is performed since no significant effect isobtained even if the filtering process is performed.

[0115] Next, it moves on to a step S204, and refers to the pointer forthe subject pixel *sx and reads the U data of the pixel data of thesubject pixels v₄ to v₇ of the second subject pixel string L_(SR), andmoves on to a step S206.

[0116] Next, it moves on to the step S206, and refers to the pointer forthe subject pixel *sx and reads V data showing a V component of thepixel data of the subject pixels v₀ to v₃ of the first subject pixelstring L_(SL). As this embodiment constitutes the V data as one piece ofdata for every two pixels, it reads the V data of the subject pixels v₀and v₁ and the V data of the subject pixels v₂ and v₈ in the step S206.Moreover, as for the V data, priority is given to acceleration of theprocessing and no deblock filtering process is performed since nosignificant effect is obtained even if the filtering process isperformed.

[0117] Next, it moves onto a step S208, and refers to the pointer forthe subject pixel *sx and reads the V data of the pixel data of thesubject pixels v₄ to v₇ of the second subject pixel string L_(SR), andmoves on to a step S210.

[0118] In the step S210, it judges as to the reproduced image whether ornot it is a first reading, and if it judges that it is the first reading(Yes), it moves on to a step S212, and refers to the pointer for thesubject pixel *sx and reads Y data showing a Y component of the pixeldata of the subject pixels v₀ to V₃ of the first subject pixel stringL_(SL). As this embodiment constitutes the Y data as one piece of dataper pixel, it reads the V data of the subject pixels v₀ and v₃ in thestep S212.

[0119] Next, it moves on to a step S214, and refers to the pointer forthe vertical reference pixel *cy to read the Y data of the pixel data ofthe subject pixels r_(y0) to r_(y3) of the first vertical referencepixel string L_(YL), and then moves on to a step S216 and refers to thepointer for the vertical reference pixel *cy to read the Y data of thepixel data of the subject pixels r_(y4) to r_(y7) of the second verticalreference pixel string L_(YR), and moves on to a step S218.

[0120] In the step S218, it refers to the pointer for the subject pixel*sx and reads the Y data of the pixel data of the subject pixels v₄ tov₇ of the second subject pixel string L_(SR), and moves onto a step S220and refers to the pointer for the subject pixel *sx and reads the Y dataof the pixel data of the subject pixels (including the reference pixelr_(x1)) of the second horizontal reference pixel string L_(XR) so as tofinish the series of processes and have the original process returned.

[0121] On the other hand, in the step S210, in the case where it judgesthat the reproduced image is read for the second time onward (No), itmoves on to a step S222, and copies in the buffer the pixel data of thesecond horizontal reference pixel string L_(XR) read in the step S220last time as the pixel data of the subject pixels v₀ to v₃ of the firstsubject pixel string L_(SL), and moves on to a step S224, and copies inthe buffer the data related to the rightmost pixel of the pixel data ofthe second subject pixel string L_(SR) read in the step S220 last timeas the pixel data of a reference pixel r_(x0) of the first horizontalreference pixel string L_(XL), and moves on to the step S214.

[0122] Next, the first horizontal filtering process will be described indetail by referring to FIG. 7. FIG. 7 is a flowchart showing the firsthorizontal filtering process.

[0123] The first horizontal filtering process is the process ofperforming the horizontal filter operation as to the subject pixels v₀to v₃ of the first subject pixel string L_(SL), and if performed in thepost filter portion 40, it first moves on to a step S300 as shown inFIG. 7.

[0124] In the step S300, it judges, by the pixel data reading process,whether or not the Y data of the subject pixels v₀ to v₃ of the firstsubject pixel string L_(SL) and the reference pixel r_(x0) of the firsthorizontal reference pixel string L_(XL) has become available in thebuffer, and if it judges that the pixel data has become available byreading or copying (Yes), it moves on to steps S302 to S312, but if itjudges otherwise (No), it waits in the step S300 until the pixel databecomes available.

[0125] The steps S302 to S312 are the processes simultaneously performedas a pipeline process in the post filter portion 40.

[0126] In the step S302, it calculates a judgment value L₀ of whether ornot an absolute value of the difference between the value of the subjectpixel v₀ and the value of the reference pixel r_(x0) is exceeding aconstant value C₁ by the following equation (37), and moves on to stepsS314, S320, S326 and S332. Here, the constant value C₁ is set at “32”with the maximum value of the pixel as “255.” It is because, if theconstant value C₁ is set to be excessively larger than “32,” the filterprocessing is performed without necessity and throughput increases sothat the filter processing can no longer be accelerated, and inversely,if set to be excessively smaller than “32,” the filter processing is notperformed even if it should be performed so that an effective filtereffect cannot be expected.

L ₀=(|v ₀ −r _(x0) |␣C ₁)  (37)

[0127] In the above equation (37), L₀ is “1” when the computing equationin the parentheses holds, and is “0” when it does not hold. Hereafter,it is the same in the steps S304, S306, S404, S413, S420, S429, S502,S504, S506, S604, S613, S620 and S629.

[0128] In the step S304, it calculates a judgment value L₁ of whether ornot the absolute value of the difference between the value of thesubject pixel v₁ and the value of the subject pixel v₀ is exceeding aconstant value C₂ is calculated by the following equation (38), andmoves on to steps S314, S320, S326 and S332. Here, the constant value C₂is set at “16” with the maximum value of the pixel as “255.” It isbecause, if the constant value C₂ is set to be excessively larger than“16,” the filter processing is performed without necessity and thethroughput increases so that the filter processing can no longer beaccelerated, and inversely, if set to be excessively smaller than “16,”the filter processing is not performed even if it should be performed sothat the effective filter effect cannot be expected.

L ₁=(|v ₁ −v ₀ |≦C ₂)  (38)

[0129] In the step S306, it calculates a judgment value L₂ Of whether ornot the absolute value of the difference between the value of thesubject pixel v₂ and the value of the subject pixel v₁ is exceeding theconstant value C₂ is calculated by the following equation (39), andmoves on to the steps S314, S320, S326 and S332.

L ₂=(|v ₂ −v ₁ |≦C ₂)  (39)

[0130] In the step S308, it calculates the value v_(x0)″ after thehorizontal filter operation as to the subject pixel v₀ by the aboveequation (1), and moves on to the steps S314, S320, S326 and S332. Inthe step S308, it sets the value to be set as v_(xo′) of the aboveequation (1) at v_(x0)″.

[0131] In the step S310, it calculates the value v_(x1)″ after thehorizontal filter operation as to the subject pixel v₁ by the aboveequation (2), and moves on to the steps S314, S320, S326 and S332. Inthe step S310, it sets the value to be set as v_(x1)′ of the aboveequation (2) at v_(x1)″.

[0132] In the step S312, it calculates the value v_(x2)″ after thehorizontal filter operation as to the subject pixel v₂ by the aboveequation (3), and moves on to the steps S314, S320, S326 and S332. Inthe step S312, it sets the value to be set as v_(x2)′ of the aboveequation (3) at v_(x)″.

[0133] The steps S314, S320, S326 and S332 are the processessimultaneously performed as the pipeline process in the post filterportion 40.

[0134] In the step S314, it judges whether or not the judgment value L₀is “1,” and if it judges that the judgment value L₀ is “1” (Yes), itmoves on to the steps S316 and sets v_(x0)″ as a new value v_(x0)′ ofthe subject pixel v₀ so as to finish the series of processes and havethe original process returned.

[0135] In the step S320, it judges whether or not an AND of the judgmentvalue L₀ and L₁ is “1,” and if it judges that the AND thereof is “1”(Yes), it moves on to the steps S322 and sets v_(x1)″ as a new valuev_(x1)′ of the subject pixel v₁ so as to finish the series of processesand have the original process returned.

[0136] In the step S326, it judges whether or not an AND of the judgmentvalue L₀ to L₂ is “1,” and if it judges that the AND thereof is “1”(Yes), it moves on to the steps S328 and sets v_(x2)″ as a new valuev_(x2)′ of the subject pixel v₂ so as to finish the series of processesand have the original process returned.

[0137] In the step S332, it sets v₃ as a new value v_(x3)′ of thesubject pixel v₃ so as to finish the series of processes and have theoriginal process returned.

[0138] On the other hand, in the step S314, in the case where it judgesthat the judgment value L₀ is “0” (No), it moves on to a step S318, andsets v₀ as a new value v_(x0)′ of the subject pixel v₀ so as to finishthe series of processes and have the original process returned.

[0139] On the other hand, in the step S320, in the case where it judgesthat the AND of the judgment values L₀ and L₁ is “0” (No), it moves onto the step S324, and sets v₁ as a new value v_(x1)′ of the subjectpixel v₁ so as to finish the series of processes and have the originalprocess returned.

[0140] On the other hand, in the step S326, in the case where it judgesthat the AND of the judgment values L₀ to L₂ is “0” (No), it moves on tothe step S330, and sets v₂ as a new value v_(x2)′ of the subject pixelv₂ so as to finish the series of processes and have the original processreturned.

[0141] Next, the first vertical filtering process will be described indetail by referring to FIG. 8. FIG. 8 is a flowchart showing the firstvertical filtering process.

[0142] The first vertical filtering process is the process of performingthe vertical filter operation as to the subject pixels v₀ to v₃ of thefirst subject pixel string L_(SL), and if performed in the post filterportion 40, it first moves on to a step S400 as shown in FIG. 8.

[0143] In the step S400, it judges, by the pixel data reading process,whether or not the Y data of the reference pixels r_(y0) to r_(y3) ofthe first vertical reference pixel string L_(YL) has become available inthe buffer, and if it judges that the pixel data has become available byreading or copying (Yes), it moves on to a step S402, but if it judgesotherwise (No), it waits in the step S400 until the pixel data becomesavailable.

[0144] In the step S402, it judges, by the first vertical filteringprocess, whether or not the horizontal filter processing has beencompleted as to the subject pixels v₀ to v₃ of the first subject pixelstring L_(SL), and if it judges that the filter processing has beencompleted (Yes), it moves on to steps S404 and S406, but if it judgesotherwise (No), it moves on to the step S400.

[0145] The steps S404 and S406 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0146] In the step S404, it calculates the judgment value L₀ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x0)′ and the value of the reference pixel r_(y0) isexceeding a constant value C₂ by the following equation (40), and moveson to a step S408. While the constant value C₂ is used here, theconstant value C₁ is used instead in the case where the first subjectpixel string L_(SL) is positioned in the highest step or in the loweststep in the subject block 10. Hereafter, it is the same in the stepsS413, S420 and S429.

L ₀=(|v _(x0) ′−r _(y0) |≦C ₂)  (40)

[0147] In the step S406, it calculates the value v_(y0)″ after thevertical filter operation as to the subject pixel v_(x0)′ by the aboveequations (29) to (36) according to the vertical position of the firstsubject pixel string L_(SL) in the subject block 10, and moves on to thesteps S408. In the step S406, it sets the value to be set as v_(y0)′ ofthe above equations (29) to (36) at v_(y0)″.

[0148] In the step S408, it judges whether or not the judgment value L₀is “1,” and if it judges that the judgment value L₀ is “1” (Yes), itmoves on to the step S410 and sets v_(y0)″ as a new value v_(y0)′ of thesubject pixel v₀, moves on to the step S411 and outputs the U data and Vdata of the subject pixels v₀ and v₁ to the RGB converting portion 42,and moves on to a step S412 to output the Y data v_(y0)″ of the subjectpixel v₀ to the RGB converting portion 42, and then moves on to thesteps S413 and S414.

[0149] The steps S413 and S414 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0150] In the step S413, it calculates the judgment value L₁ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x1)′ and the value of the reference pixel r_(y1) isexceeding the constant value C₂ by the following equation (41), andmoves on to a step S416.

L ₁=(v _(x1) ′−r _(y1) |≦C ₂)  (41)

[0151] In the step S414, it calculates the value v_(y1)″ after thevertical filter operation as to the subject pixel v_(x1)′ by the aboveequations (29) to (36) according to the vertical position of the firstsubject pixel string L_(SL) in the subject block 10, and moves on to thesteps S416. In the step S414, it sets the value to be set as v_(y1)′ ofthe above equations (29) to (36) at v_(y1)″.

[0152] In the step S416, it judges whether or not the judgment value L₁is “1,” and if it judges that the judgment value L₁ is “1” (Yes), itmoves on to the step S418 and sets v_(y1) as a new value v_(y1)′ of thesubject pixel v₁, moves on to the step S419 and outputs the Y datav_(y1)″ of the subject pixel v₁ to the RGB converting portion 42, andmoves on to the steps S420 and S422.

[0153] The steps S420 and S422 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0154] In the step S420, it calculates the judgment value L₂ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x2)′ and the value of the reference pixel r_(y2) isexceeding the constant value C₂ by the following equation (42), andmoves on to a step S424.

L ₂=(|v _(x2) ′−r _(y2) |≦C ₂)  (42)

[0155] In the step S422, it calculates the value v_(y2)″ after thevertical filter operation as to the subject pixel v_(x2)′ by the aboveequations (29) to (36) according to the vertical position of the firstsubject pixel string L_(SL) in the subject block 10, and moves on to thesteps S424. In the step S422, it sets the value to be set as v_(y2)′ ofthe above equations (29) to (36) at v_(y2)″.

[0156] In the step S424, it judges whether or not the judgment value L₂is “1,” and if it judges that the judgment value L₂ is “1” (Yes), itmoves on to the step S426 and sets v_(y2)″ as a new value v_(y2)′ of thesubject pixel v₂, moves on to the step S427 and outputs the U data and Vdata of the subject pixels v₂ and v₃ to the RGB converting portion 42,and moves on to a step S428 to output the Y data v_(y2)″ of the subjectpixel v₂ to the RGB converting portion 42, and then moves on to thesteps S429 and S430.

[0157] The steps S429 and S430 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0158] In the step S429, it calculates the judgment value L₃ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x3)′ and the value of the reference pixel r_(y3) isexceeding the constant value C₂ by the following equation (43), andmoves on to a step S432.

L ₃=(|v _(x3) ′−r _(y3) |≦C ₂)  (43)

[0159] In the step S430, it calculates the value v_(y3)″ after thevertical filter operation as to the subject pixel v_(x3)′ by the aboveequations (29) to (36) according to the vertical position of the firstsubject pixel string L_(SL) in the subject block 10, and moves on to thesteps S432. In the step S430, it sets the value to be set as v_(y3)′ ofthe above equations (29) to (36) at v_(y3)″.

[0160] In the step S432, it judges whether or not the judgment value L₃is “1,” and if it judges that the judgment value L₃ is “1” (Yes), itmoves on to a step S434 and sets v_(y3)″ as a new value v_(y3)′ of thesubject pixel v₃, moves on to a step S446 and outputs the Y data v_(y3)″of the subject pixel V₃ to the RGB converting portion 42 so as to finishthe series of processes and have the original process returned.

[0161] On the other hand, in the step S432, in the case where it judgesthat the judgment values L₃ is “0” (No), it moves on to a step S448, andsets v_(x3)′ as a new value v_(y3)′ of the subject pixel v₃, and moveson to a step S446.

[0162] On the other hand, in the step S424, in the case where it judgesthat the judgment value L₂ is “0” (No), it moves on to a step S450, andsets v_(x2)′ as a new value v_(y2)′ of the subject pixel v₂, and moveson to the step S427.

[0163] On the other hand, in the step S416, in the case where it judgesthat the judgment value L₁ is “0” (No), it moves on to a step S452, andsets v_(x1)′ as a new value v_(y1)′ of the subject pixel v₁, and moveson to the step S419.

[0164] On the other hand, in the step S408, in the case where it judgesthat the judgment value L₀ is “0” (No), it moves on to a step S454, andsets v_(x0)′ as a new value v_(y0)′ of the subject pixel v₀, and moveson to the step S411.

[0165] Next, the second horizontal filtering process will be describedin detail by referring to FIG. 9. FIG. 9 is a flowchart showing thesecond horizontal filtering process.

[0166] The second horizontal filtering process is the process ofperforming the horizontal filter operation as to the subject pixels v₄to v₇ of the second subject pixel string L_(SR), and if performed in thepost filter portion 40, it first moves on to a step S500 as shown inFIG. 9.

[0167] In the step S500, it judges, by the pixel data reading process,whether or not the Y data of the subject pixels v₄ to v₇ of the secondsubject pixel string L_(SR) and the reference pixel r_(x1) of the secondhorizontal reference pixel string L_(XR) has become available in thebuffer, and if it judges that the pixel data has become available byreading or copying (Yes), it moves on to step S502 to S512, but if itjudges otherwise (No), it waits in the step S500 until the pixel databecomes available.

[0168] The steps S502 and S512 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0169] In the step S502, it calculates the judgment value L₇ of whetheror not the absolute value of the difference between the value of thesubject pixel v₇ and the value of the reference pixel r_(x1) isexceeding the constant value C₁ by the following equation (44), andmoves on to the steps S514, S520, S526 and S532.

L ₇=(|v ₇ −r _(x1) |≦C ₁)  (44)

[0170] In the step S504, it calculates the judgment value L₆ of whetheror not the absolute value of the difference between the value of thesubject pixel v₆ and the value of the subject pixel V₇ is exceeding theconstant value C₂ by the following equation (45), and moves on to thesteps S514, S520, S526 and S532.

L ₆=(|v ₆ −v ₇ |≦C ₂)  (45)

[0171] In the step S506, it calculates the judgment value L₅ of whetheror not the absolute value of the difference between the value of thesubject pixel v₅ and the value of the subject pixel v₆ is exceeding theconstant value C₂ by the following equation (46), and moves on to thesteps S514, S520, S526 and S532.

L ₅=(|v ₅ −v ₆ |≦C ₂)  (46)

[0172] In the step S508, it calculates the value v_(x7)″ after thehorizontal filter operation as to the subject pixel v₇ by the aboveequation (8), and moves on to the steps S514, S520, S526 and S532. Inthe step S508, it sets the value to be set as v_(x7)′ of the aboveequation (8) at v_(x7)″.

[0173] In the step S510, it calculates the value v_(x6)″ after thehorizontal filter operation as to the subject pixel v6 by the aboveequation (7), and moves on to the steps S514, S520, S526 and S532. Inthe step S510, it sets the value to be set as v_(x6)′ of the aboveequation (7) at v_(x6)″.

[0174] In the step S512, it calculates the value v_(x5)″ after thehorizontal filter operation as to the subject pixel v₅ by the aboveequation (6), and moves on to the steps S514, S520, S526 and S532. Inthe step S512, it sets the value to be set as v_(x5)′ of the aboveequation (6) at v_(x5)″.

[0175] The steps S514, S520, S526 and S532 are the processessimultaneously performed as the pipeline process in the post filterportion 40.

[0176] In the step S514, it judges whether or not the judgment value L₇is “1,” and if it judges that the judgment value L₇ is “1” (Yes), itmoves on to the step S516 and sets v_(x7)″ as a new value v_(x7′) of thesubject pixel v₇ so as to finish the series of processes and have theoriginal process returned.

[0177] In the step S520, it judges whether or not the AND of thejudgment values L₇ and L₆ is “1,” and if it judges that the AND thereofis “1” (Yes), it moves on to the step S522 and sets v_(x6)″ as a newvalue v_(x6)′ of the subject pixel v₆ so as to finish the series ofprocesses and have the original process returned.

[0178] In the step S526, it judges whether or not the AND of thejudgment values L₇ to L₅ is “1,” and if it judges that the AND thereofis “1” (Yes), it moves on to the step S528 and sets v_(x5)″ as a newvalue v_(x5′)′ of the subject pixel v₅ so as to finish the series ofprocesses and have the original process returned.

[0179] In the step S532, it sets v₄ as a new value v_(x4)′ of thesubject pixel v₄ so as to finish the series of processes and have theoriginal process returned.

[0180] On the other hand, in the step S514, in the case where it judgesthat the judgment value L₇ is “0” (No), it moves on to the step S518 andsets v₇ as a new value v_(x7)′ of the subject pixel v₇ so as to finishthe series of processes and have the original process returned.

[0181] On the other hand, in the step S520, in the case where it judgesthat the AND of the judgment values L₇ and L₆ is “0” (No), it moves onto the step S524, and sets v6 as a new value v_(x6)′ of the subjectpixel v₆ so as to finish the series of processes and have the originalprocess returned.

[0182] On the other hand, in the step S526, in the case where it judgesthat the AND of the judgment values L₇ to L₅ is “0” (No), it moves on tothe step S530, and sets v₅ as a new value v_(x5)′ of the subject pixelv₅ so as to finish the series of processes and have the original processreturned.

[0183] Next, the second vertical filtering process will be described indetail by referring to FIG. 10. FIG. 10 is a flowchart showing thesecond vertical filtering process.

[0184] The second vertical filtering process is the process ofperforming the vertical filter operation as to the subject pixels v₄ tov₇ of the second subject pixel string L_(SR), and if performed in thepost filter portion 40, it first moves on to a step S600 as shown inFIG. 10.

[0185] In the step S600, it judges, by the pixel data reading process,whether or not the Y data of the reference pixels r_(y4) to r_(y7) ofthe second vertical reference pixel string L_(YR) has become availablein the buffer, and if it judges that the pixel data has become availableby reading or copying (Yes), it moves on to a step S602, but if itjudges otherwise (No), it waits in the step S600 until the pixel databecomes available.

[0186] In the step S602, it judges, by the second horizontal filteringprocess, whether or not the horizontal filter processing has beencompleted as to the subject pixels v₄ to v₇ of the second subject pixelstring L_(SR), and if it judges that the filter processing has beencompleted (Yes), it moves on to the steps S604 and S606, but if itjudges otherwise (No), it moves on to the step S600.

[0187] The steps S604 and S606 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0188] In the step S604, it calculates the judgment value L₇ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x7)′ and the value of the reference pixel r_(y7) isexceeding a constant value C₂ by the following equation (47), and movesonto a step S608. While the constant value C₂ is used here, the constantvalue C₁ is used instead in the case where the second subject pixelstring L_(SR) is positioned in the highest step or in the lowest step inthe subject block 10. Hereafter, it is the same in the steps S613, S620and S629.

L ₇=(|v _(x7) ′−r _(y7) |≦C ₂)  (47)

[0189] In the step S606, it calculates the value v_(y7)″ after thevertical filter operation as to the subject pixel v_(x7)′ by the aboveequations (29) to (36) according to the vertical position of the secondsubject pixel string L_(SR) in the subject block 10, and moves on to thestep S608. In the step S606, it sets the value to be set as v_(y7)′ ofthe above equations (29) to (36) at v_(y7)″.

[0190] In the step S608, it judges whether or not the judgment value L₇is “1,” and if it judges that the judgment value L₇ is “1” (Yes), itmoves on to the step S610 and sets v_(y7)″ as a new value v_(y7)′ of thesubject pixel v₇, moves on to the step S611 and outputs the U data and Vdata of the subject pixels v₇ and v₆ to the RGB converting portion 42,and moves on to a step S612 to output the Y data v_(y7)″ of the subjectpixel v₇ to the RGB converting portion 42, and then moves on to thesteps S613 and S614.

[0191] The steps S613 and S614 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0192] In the step S613, it calculates the judgment value L₆ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x6)′ and the value of the reference pixel r_(y6) isexceeding the constant value C₂ by the following equation (48), andmoves on to a step S616.

L ₆=(|v _(x6) ′−r _(y6) |≦C ₂)  (48)

[0193] In the step S614, it calculates the value v_(y6)″ after thevertical filter operation as to the subject pixel v_(x6)′ by the aboveequations (29) to (36) according to the vertical position of the secondsubject pixel string L_(SR) in the subject block 10, and moves on to thesteps S616. In the step S614, it sets the value to be set as v_(y6)′ ofthe above equations (29) to (36) at v_(y6)″.

[0194] In the step S616, it judges whether or not the judgment value L₆is “1,” and if it judges that the judgment value L₆ is “1” (Yes), itmoves on to the step S618 and sets v_(y6)″ as a new value v_(y6)′ of thesubject pixel v₆, moves on to the step S619 and outputs the Y datav_(y6)″ of the subject pixel v₆ to the RGB converting portion 42, andmoves on to the steps S620 and S622.

[0195] The steps S620 and S622 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0196] In the step S620, it calculates the judgment value L₅ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x5)′ and the value of the reference pixel r_(y5) isexceeding the constant value C₂ by the following equation (49), andmoves on to a step S624.

L ₅=(|v _(x5) ′−r _(y5) |≦C ₂)  (49)

[0197] In the step S622, it calculates the value v_(y5)″ after thevertical filter operation as to the subject pixel v_(x5)′ by the aboveequations (29) to (36) according to the vertical position of the secondsubject pixel string L_(SR) in the subject block 10, and moves on to thestep S624. In the step S622, it sets the value to be set as v_(y5)′ ofthe above equations (29) to (36) at v_(y5)″.

[0198] In the step S624, it judges whether or not the judgment value L₅is “1,” and if it judges that the judgment value L₅ is “1” (Yes), itmoves on to the step S626 and sets v_(y5)″ as a new value v_(y5)′ of thesubject pixel v₅, moves on to the step S627 and outputs the U data and Vdata of the subject pixels v₅ and V₄ to the RGB converting portion 42,and moves on to a step S628 to output the Y data v_(y5)″ of the subjectpixel v₅ to the RGB converting portion 42, and then moves on to thesteps S629 and S630.

[0199] The steps S629 and S630 are the processes simultaneouslyperformed as the pipeline process in the post filter portion 40.

[0200] In the step S629, it calculates the judgment value L₄ of whetheror not the absolute value of the difference between the value of thesubject pixel v_(x4)′ and the value of the reference pixel r_(y4) isexceeding the constant value C₂ by the following equation (50), andmoves on to a step S632.

L ₄=(|v _(x4) ′−r _(y4) |≦C ₂)  (50)

[0201] In the step S630, it calculates the value v_(y4)″ after thevertical filter operation as to the subject pixel v_(x4)′ by the aboveequations (29) to (36) according to the vertical position of the secondsubject pixel string L_(SR) in the subject block 10, and moves on to thesteps S632. In the step S630, it sets the value to be set as v_(y4)′ ofthe above equations (29) to (36) at v_(y4)″.

[0202] In the step S632, it judges whether or not the judgment value L₄is “1,” and if it judges that the judgment value L₄ is “1” (Yes), itmoves on to a step S634 and sets v_(y4)″ as a new value v_(y4)′ of thesubject pixel v₄, moves on to a step S646 and outputs the Y data v_(y4)″of the subject pixel v₄ to the RGB converting portion 42 so as to finishthe series of processes and have the original process returned.

[0203] On the other hand, in the step S632, in the case where it judgesthat the judgment values L₄ is “0” (No), it moves on to a step S648, andsets v_(x4)′ as a new value v_(y4)′ of the subject pixel v₄, and moveson to a step S646.

[0204] On the other hand, in the step S624, in the case where it judgesthat the judgment value L₅ is “0” (No), it moves on to a step S650, andsets v_(x5)′ as a new value v_(y5)′ of the subject pixel v₅, and moveson to the step S627.

[0205] On the other hand, in the step S616, in the case where it judgesthat the judgment value L₆ is “0” (No), it moves on to a step S652, andsets v_(x6)′ as a new value v_(y6)′ of the subject pixel v₆, and moveson to the step S619.

[0206] On the other hand, in the step S608, in the case where it judgesthat the judgment value L₇ is “0” (No), it moves on to a step S654, andsets v_(x7)′ as a new value v_(y7)′ of the subject pixel v₇, and moveson to the step S611.

[0207] Next, the pointer updating process of the above step S106 will bedescribed in detail by referring to FIG. 11. FIG. 11 is a flowchartshowing the pointer updating process.

[0208] If performed in the above step S106, the pointer updating processfirst moves on to the step S700 as shown in FIG. 11. Moreover, in thefollowing steps, the number of horizontal pixels of the reproduced imageis X, and the vertical coordinate of first subject pixel string L_(SL)and second subject pixel string L_(SR) in the subject block 10 is y.

[0209] In the step S700, it adds an equivalent of eight pixels to thevalues of the pointer for the subject pixel *sx and the pointer for thevertical reference pixel *cy. This addition moves the subject block 10and the pointer for the vertical reference pixel *cy to the right by oneblock. In the case where the subject block 10 before the addition is therightmost block in the reproduced image, however, they move to theleftmost block lower by one step. As this move only adds a predeterminednumber, it can be implemented just by an adder in case of constitutingit by the hardware.

[0210] In the step S702, it judges whether or not the filtering processhas been finished as to the subject pixels of the rightmost block in thereproduced image, and if it judges that the filtering process has beenfinished (Yes), it moves on to the step S704, but if it judges otherwise(No), it finishes the series of processes and has the original processreturned.

[0211] In the step S704, it calculates the remainder of dividing y by“8,” moves on to the step S706 and judges whether or not the calculatedremainder is “0,” and if it judges that the remainder is “0” (Yes), itmoves onto the step S708 and subtracts 2X from the pointer for thevertical reference pixel *cy so as to finish the series of processes andhave the original process returned. This subtraction moves the pointerfor the vertical reference pixel *cy to the lowest step of the verticalreference block 16 adjacent to it above the subject block 10. As thismove only doubles X and subtracts it, it can be implemented just by theshifter and adder in case of constituting it by the hardware.

[0212] On the other hand, in the step S706, in the case where it judgesthat the calculated remainder is not “0” (No), it moves on to a stepS710 and judges whether or not the calculated remainder is “4,” and ifit judges that the remainder is “4” (Yes), it moves on to the step S712and adds 8X to the value of the pointer for the vertical reference pixel*cy so as to finish the series of processes and have the originalprocess returned. This addition moves the pointer for the verticalreference pixel *cy to the highest step of the vertical reference block18 adjacent to it below the subject block 10. As this move onlymultiplies X by 8 and adds it, it can be implemented just by the shifterand adder in case of constituting it by the hardware.

[0213] On the other hand, in the step S710, in the case where it judgesthat the calculated remainder is not “4” (No), it moves on to a stepS714, and subtracts X from the pointer for the vertical reference pixel*cy so as to finish the series of processes and have the originalprocess returned. This subtraction moves the pointer for the verticalreference pixel *cy to the lowest step of the vertical reference block16 adjacent to it above the subject block 10. As this move onlysubtracts X, it can be implemented just by the adder in case ofconstituting it by the hardware.

[0214] Next, operation of this embodiment will be described by referringto the drawings. FIG. 12 is a time chart showing the process ofperforming the deblock filtering process. Hereafter, each of times t₀ tot₁₂ indicates the time when a predetermined time T equivalent to 1 clockof the post filter portion 40 elapsed each time. To be more specific,each of the times t₀ to t₁₂ indicates the time when the predeterminedtime T elapsed from the time immediately preceding it.

[0215] First, at the time t₀, through the step S100, the post filterportion 40 sets as the pointer for the subject pixel *sx the address atwhich the pixel data of the leftmost pixel of the first subject pixelstring L_(SL) to be processed first in the VRAM 35 is stored, and setsas the pointer for the vertical reference pixel *cy the address at whichthe pixel data of the leftmost pixel of the first vertical referencepixel string L_(YL) to be processed first in the VRAM 35 is storedrespectively. And, through the step S102, it refers to the pointer forthe subject pixel *sx and the pointer for the vertical reference pixel*cy, and outputs the read request to read the first subject pixel stringL_(SL), second subject pixel string L_(SR), second horizontal referencepixel string L_(XR), first vertical reference pixel string L_(YL) andsecond vertical reference pixel string L_(YR).

[0216] If the read request is outputted, at the time t₁, through thesteps S202 and S204, the U data of the subject pixels v₀ and v₁, the Udata of the subject pixels v₂ and v₃, the U data of the subject pixelsv₄ and v₅ and the U data of the subject pixels v₆ and v₇ are read to theVRAM 35.

[0217] Subsequently, at the time t₂, through the steps S206 and S208,the V data of the subject pixels v₀ and v₁, the V data of the subjectpixels v₂ and v₃, the V data of the subject pixels v₄ and v₅ and the Vdata of the subject pixels v₆ and v₇ are read to the VRAM 35.

[0218] On the other hand, at the same time t₂, through the steps S222and S224, it copies in the buffer the pixel data of the secondhorizontal reference pixel string L_(XR) read last time as the pixeldata of the subject pixels v₀ to v₃ of the first subject pixel stringL_(SL), and copies in the buffer the data related to the rightmost pixelof the pixel data of the second subject pixel string L_(SR) read lasttime as the pixel data of the reference pixel r_(x0) of the firsthorizontal reference pixel string L_(XL) as shown in FIG. 13. FIG. 13 isa diagram for explaining the case of copying and reutilizing the pixeldata.

[0219] Subsequently, at the time t₃, through the step S214, it refers tothe pointer for the vertical reference pixel *cy, and the Y data of thepixel data of the subject pixels r_(y0) to r_(y3) of the first verticalreference pixel string L_(YL) is read to the VRAM 35.

[0220] On the other hand, at the same time t₃, the Y data of the subjectpixels v₀ to v₃ of the first subject pixel string L_(SL) and thereference pixel r_(x0) of the first horizontal reference pixel stringL_(XL) has become available in the buffer, and so the judgment values L₀to L₂ are calculated by the above equations (37) to (39) and v_(x0)″ tov_(x2)″ are calculated by the above equations (1) to (3) through thesteps S302 to S312. At this time, if the judgment values L₀ is “1,” itsets v_(x0)″ as a new value v_(x0)′ of the subject pixel v₀ through thesteps S316, and if the judgment values L₀ is “0,” it sets v₀ as a newvalue v_(x0)′ of the subject pixel v₀ through the steps S318. Inaddition, if the AND of the judgment values L₀ and L₁ is “1,” it setsv_(x1)″ as a new value v_(x1)′ of the subject pixel v₁ through the stepsS322, and if the AND is “0,” it sets v₁ as a new value v_(x1)′ of thesubject pixel v₁ through the steps S324. Moreover, if the AND of thejudgment values L₀ to L₂ is “1,” it sets v_(x2)″ as a new value v_(x2)′of the subject pixel v₂ through the steps S328, and if the AND is “0,”it sets v₂ as a new value v_(x2)′ of the subject pixel v₂ through thesteps S330. In addition, it sets v₃ as a new value v_(x3)′ of thesubject pixel v₃ through the steps S332.

[0221] Subsequently, at the time t₄, through the step S216, it refers tothe pointer for the vertical reference pixel *cy, and the Y data of thepixel data of the reference pixels r_(y4) to r_(y7) of the secondvertical reference pixel string L_(YR) is read to the VRAM 35.

[0222] On the other hand, at the same time t₄, the Y data of thereference pixels r_(y0) to r_(y3) of the first vertical reference pixelstring L_(YL) has become available in the buffer and the horizontalfiltering process has been completed as to the subject pixels v₀ to v₃of the first subject pixel string L_(SL), and so the judgment value L₀is calculated by the above equation (40) through the steps S404 andS406, and v_(y0)″ is calculated by one of the above equations (29) to(36). At this time, if the judgment values L₀ is “1,” it sets v_(y0)″ asa new value v_(y0)′ of the subject pixel v₀ through the steps S410, andif the judgment values L₀ is “0,” it sets v_(x0) as a new value v_(y0)′of the subject pixel v₀ through the step S454. And, through the stepsS411 and 412, it outputs the U data and V data of the subject pixels v₀and v₁ to the RGB converting portion 42, and outputs the Y data v_(y0)″of the subject pixel v₀ to the RGB converting portion 42. The deblockfiltering process as to the subject pixel v₀ is thereby completed.

[0223] Subsequently, at the time t₅, through the step S218, it refers tothe pointer for the subject pixel *sx, and the Y data of the pixel dataof the subject pixels v₄ to v₇ of the second subject pixel string L_(SR)is read to the VRAM 35.

[0224] On the other hand, at the same time t₅, through the steps S413and S414, the judgment value L₁ is calculated by the above equation(41), and v_(y1)″ is calculated by one of the above equations (29) to(36). At this time, if the judgment values L₁ is “1,” it sets v_(y1)″ asa new value v_(y1)′ of the subject pixel v₁ through the step S418, andif the judgment values L₁ is “0,” it sets v_(x1) as a new value v_(y1)′of the subject pixel v₁ through the step S452. And, through the stepsS419, it outputs the Y data v_(y1)″ of the subject pixels v₁ to the RGBconverting portion 42. The deblock filtering process as to the subjectpixel v₁ is thereby completed.

[0225] Subsequently, at the time t₆, through the step S220, it refers tothe pointer for the subject pixel *sx, and the Y data of the pixel dataof the subject pixels (including the reference pixel r_(x1)) of thesecond horizontal reference pixel string L_(XR) is read to the VRAM 35.

[0226] On the other hand, at the same time t₆, the judgment value L₂ iscalculated by the above equation (42) through the steps S420 and S422,and v_(y2)″ is calculated by one of the above equations (29) to (36). Atthis time, if the judgment values L₂ is “1,” it sets v_(y2)″ as a newvalue v_(y2)′ of the subject pixel v₂ through the step S426, and if thejudgment values L₂ is “0,” it sets v_(x2)′ as a new value v_(y2)′ of thesubject pixel v₂ through the step S450. And, through the steps S427 andS428, it outputs the U data and V data of the subject pixels v₂ and v₃to the RGB converting portion 42, and outputs the Y data v_(y2)″ of thesubject pixel v₂ to the RGB converting portion 42. The deblock filteringprocess as to the subject pixel v₂ is thereby completed.

[0227] Subsequently, at the time t₇, the judgment value L₃ is calculatedby the above equation (43) through the steps S429 and S430, and v_(y3)″is calculated by one of the above equations (29) to (36). At this time,if the judgment value L₃ is “1,” it sets v_(y3)″ as a new value v_(y3)′of the subject pixel v₃ through the step S434, and if the judgment valueL₃ is “0,” it sets v_(x3)′ as a new value v_(y3)′ of the subject pixelv₃ through the step S448. And, through the step S446, it outputs the Ydata v_(y3)″ of the subject pixels v₃ to the RGB converting portion 42.The deblock filtering process as to the subject pixel v₃ is therebycompleted.

[0228] On the other hand, at the same time t₇, the Y data of the subjectpixels v₄ to v₇ of the second subject pixel string L_(SR) and thereference pixel r_(x1) of the second horizontal reference pixel stringL_(XR) has become available in the buffer, and so the judgment values L₇to L₅ are calculated by the above equations (44) to (46) and v_(x7)″ tov_(x5)″ are calculated by the above equations (8) to (6) through thesteps S502 to S512. At this time, if the judgment values L₇ is “1,” itsets v_(x7)″ as a new value v_(x7)′ of the subject pixel v₇ through thestep S516, and if the judgment values L₇ is “0,” it sets v₇ as a newvalue v_(x7)′ of the subject pixel v₇ through the step S518. Inaddition, if the AND of the judgment values L₇ and L₆ is “1,” it setsv_(x6)″ as a new value v_(x6)′ of the subject pixel v₆ through the stepS522, and if the AND is “0,” it sets v₆ as a new value v_(x6)′ of thesubject pixel v₆ through the step S524. Moreover, if the AND of thejudgment values L₇ to L₅ is “1,” it sets v_(x5)′ as a new value v_(x5)′of the subject pixel v₅ through the step S528, and if the AND is “0,” itsets v₅ as a new value v_(x5)′ of the subject pixel v₅ through the stepS530. In addition, it sets v₄ as a new value v_(x4)′ of the subjectpixel v₄ through the step S532.

[0229] Subsequently, at the same time t₈, the Y data of the referencepixels r_(y4) to r_(y7) of the second vertical reference pixel stringL_(YR) has become available in the buffer and the horizontal filteringprocess has been completed as to the subject pixels v₄ to v₇ of thesecond subject pixel string L_(SR), and so the judgment value L₇ iscalculated by the above equation (47) through the steps S604 and S606,and v_(y7)″ is calculated by one of the above equations (29) to (36). Atthis time, if the judgment values L₇ is “1,” it sets v_(y7)″ as a newvalue v_(y7)′ of the subject pixel v₇ through the step S610, and if thejudgment value L₇ is “0,” it sets v_(x7)′ as a new value v_(y7)′ of thesubject pixel v₇ through the step S654. And, through the steps S611 and612, it outputs the U data and V data of the subject pixels v₇ and v₆ tothe RGB converting portion 42, and outputs the Y data v_(y7)″ of thesubject pixel v₇ to the RGB converting portion 42. The deblock filteringprocess as to the subject pixel v₇ is thereby completed.

[0230] Moreover, at the same time t₈, through the steps S700 and S714,the pointer for the subject pixel *sx and the pointer for the verticalreference pixel *cy move to the block adjacent to the subject block 10on the right thereof, and the same process as that of the time t₀ isperformed to that block.

[0231] Next, at the same time t₈, through the steps S613 and S614, thejudgment value L₆ is calculated by the above equation (48), and v_(y6)″is calculated by one of the above equations (29) to (36). At this time,if the judgment value L6 is “1,” it sets v_(y6)″ as a new value v_(y6)′of the subject pixel v₆ through the step S618, and if the judgment valueL₆ is “0,” it sets v_(x6)′ as a new value v_(y6)′ of the subject pixelv₆ through the step S652. And, through the step S619, it outputs the Ydata v_(y6)″ of the subject pixel v₆ to the RGB converting portion 42.The deblock filtering process as to the subject pixel v₆ is therebycompleted.

[0232] Moreover, at the same time t₉, the same process as that of thetime t₁ is performed to the block adjacent to the subject block 10 onthe right thereof.

[0233] Next, at the same time t₁₀, through the steps S620 and S622, thejudgment value L₅ is calculated by the above equation (49), and v_(y5)″is calculated by one of the above equations (29) to (36). At this time,if the judgment value L₅ is “1,” it sets v_(y5)″ as a new value v_(y5)′of the subject pixel v₅ through the step S626, and if the judgment valueL₅ is “0,” it sets v_(x5)′ as a new value v_(y5)′ of the subject pixelv₅ through the step S650. And, through the steps S627 and S628, itoutputs the U data and V data of the subject pixel v₅ and v₄ to the RGBconverting portion 42, and outputs the Y data v_(y5)″ of the subjectpixel v₅ to the RGB converting portion 42. The deblock filtering processas to the subject pixel v₅ is thereby completed.

[0234] Moreover, at the same time t₁₀, the same process as that of thetime t₂ is performed to the block adjacent to the subject block 10 onthe right thereof.

[0235] Next, at the time t₁₁, through the steps S629 and S630, thejudgment value L₄ is calculated by the above equation (50), and v_(y4)″is calculated by one of the above equations (29) to (36). At this time,if the judgment value L₄ is “1,” it sets v_(y4)″ as a new value v_(y4)′of the subject pixel v₄ through the step S634, and if the judgment valueL₄ is “0,” it sets v_(x4)′ as a new value v_(y4)′ of the subject pixelv₄ through the step S648. And, through the step S646, it outputs the Ydata v_(y4)″ of the subject pixel v₄ to the RGB converting portion 42.The deblock filtering process as to the subject pixel v₄ is therebycompleted.

[0236] Moreover, at the same time t₁₁, the same process as that of thetime t₃ is performed to the block adjacent to the subject block 10 onthe right thereof.

[0237] Thus, according to this embodiment, the deblock filtering processjudges whether or not the difference values between the values of thesubject pixels v₀ to v₇ and the values of the reference pixels r_(y0) tor_(y7) are exceeding a threshold, and performs the horizontal filteroperation as to the subject pixels v₀ to v₇ based solely on the valuesof the subject pixels v₀ to v₇ and the values of the reference pixelsr_(x0) and r_(x1), determines whether or not to perform the verticalfilter operation as to the subject pixel s v₀ to v₇ based solely on theabove judgment result, and performs the vertical filter operation as tothe subject pixels v₀ to v₇ based solely on the values of the subjectpixels v₀ to v₇ and the values of the reference pixels r_(y0) to r_(y7).

[0238] Thus, in case of judging whether or not to perform the filteroperation and performing the filter operation, it just has to refer tothe subject pixels v₀ to v₇, the reference pixels r_(x0) and r_(x1) andthe reference pixels r_(y0) to r_(y7) so that the deblock filteringprocess can be simplified compared to the cases in the past. Forinstance, in case of implementing the deblock filtering process by thehardware as in this embodiment, a circuit scale can be rendered smaller,and in case of implementing the deblock filtering process by software, aprogram size can be reduced. In particular, as the pixels positioned onthe boundaries in the horizontal reference blocks 12 and 14 and thevertical reference blocks 16 and 18 are used as the reference pixelr_(x0) and r_(x1) and the reference pixels r_(y0) to r_(y7), the deblockfiltering process can be simplified without reducing the filter effectsignificantly. Furthermore, the number of times of access to the VRAM 35can be reduced so that the acceleration of processing can be expected.

[0239] Furthermore, according to this embodiment, the deblock filteringprocess tries the filter operation as to all the pixels of thereproduced image while repeating reading of the pixel data, the judgmentand the filter operation as the series of processes, and as to thesubject pixel belonging to the upper left area of the subject block 10,it uses the values of the reference pixels of the horizontal referenceblock 12 and the vertical reference block 16 adjacent to the subjectblock 10 on the left of and above it for the judgment and the filteroperation, as to the subject pixel belonging to the upper right area ofthe subject block 10, it uses the values of the reference pixels of thehorizontal reference block 14 and the vertical reference block 16adjacent to the subject block 10 on the right of and above it for thejudgment and the filter operation, as to the subject pixel belonging tothe lower left area of the subject block 10, it uses the values of thereference pixels of the horizontal reference block 12 and the verticalreference block 18 adjacent to the subject block 10 on the left of andbelow it for the judgment and the filter operation, and as to thesubject pixel belonging to the lower right area of the subject block 10,it uses the values of the reference pixels of the horizontal referenceblock 14 and the vertical reference block 18 adjacent to the subjectblock on the right of and below it for the judgment and the filteroperation.

[0240] Thus, in case of performing the deblock filtering process to thefour subject pixels v₀ to v₃, it only has to read nine pixels from theVRAM 35 so that the number of times of reading to the VRAM 35 can bereduced. In addition, in case of judging whether or not to perform thefilter operation and performing the filter operation, comparativelyadequate reference pixels are used according to the area which thesubject pixels v₀ to v₃ belong to, and so the filter effect can beimproved.

[0241] Furthermore, according to this embodiment, the deblock filteringprocess refers to the pointer for the vertical reference pixel *cy toread the pixel data to the buffer, and further adds an equivalent ofeight pixels to the value of the pointer for the vertical referencepixel *cy in each series of the processes, subtracts 2X from the valueof the pointer for the vertical reference pixel *cy when the subjectblock 10 be processed next is the leftmost block in the reproduced imageand the remainder on dividing y by “8” is “0,” adds 8X to the value ofthe pointer for the vertical reference pixel *cy when the subject block10 to be processed next is the leftmost block in the reproduced imageand the remainder on dividing y by “8” is “4,” and subtracts X from thevalue of the pointer for the vertical reference pixel *cy when thesubject block 10 to be processed next is the leftmost block in thereproduced image and the remainder on dividing y by “8” is a value otherthan “0” or “4.”

[0242] Thus, updating of the pointer for the vertical reference pixel*cy can be implemented just by the shifter and the adder so that theload can be reduced and the configuration can be rendered comparativelysimple.

[0243] Furthermore, according to this embodiment, as to each series ofprocesses, the deblock filtering process reads the second subject pixelstring L_(SR), second horizontal reference pixel string L_(XR), firstvertical reference pixel string L_(YL) and second vertical referencepixel string L_(YR) to the buffer, and also reutilizes the pixel data ofthe second subject pixel string L_(SR) and second horizontal referencepixel string L_(XR) read in the immediately preceding process as thepixel data of the first horizontal reference pixel string L_(XL) andfirst subject pixel string L_(SL) respectively, so that the judgment andthe filter operation are performed as to all the subject pixels v₀ to v₇of the first subject pixel string L_(SL) and second subject pixel stringL_(SR).

[0244] Thus, the number of times of reading to the VRAM 35 can furtherbe reduced since it is not necessary to reread the pixel data of thefirst horizontal reference pixel string L_(XL) and first subject pixelstring L_(SL) from VRAM 35.

[0245] Furthermore, according to this embodiment, in the case where afirst phase is the process of copying the pixel data of the secondsubject pixel string L_(SR) and second horizontal reference pixel stringL_(XR) read in the immediately preceding process as the pixel data ofthe first horizontal reference pixel string L_(XL) and first subjectpixel string L_(SL) respectively, a second phase is the process ofreading the pixel data of the first vertical reference pixel stringL_(YL) to the buffer, a third phase is the process of reading the pixeldata of the second vertical reference pixel string L_(YR) to the buffer,a fourth phase is the process of reading the pixel data of the secondsubject pixel string L_(SR) into the buffer, a fifth phase is theprocess of reading the pixel data of the second horizontal referencepixel string L_(XR) into the buffer, a sixth phase is the process ofperforming the horizontal filter operation as to each subject pixel ofthe first subject pixel string L_(SL) based on the pixel data of thefirst horizontal reference pixel string L_(XL), a seventh phase is theprocess of performing the judgment and the vertical filter operation asto each subject pixel of the first subject pixel string L_(SL) based onprocessing results of the sixth phase and the pixel data of the firstvertical reference pixel string L_(YL), an eighth phase is the processof performing the horizontal filter operation as to each subject pixelof the second subject pixel string L_(SR) based on the pixel data of thesecond horizontal reference pixel string L_(XR), and a ninth phase isthe process of performing the judgment and the vertical filter operationas to each subject pixel of the second subject pixel string L_(SR) basedon the processing results of the eighth phase and the pixel data of thesecond vertical reference pixel string L_(YR), the series of processesperforms the first phase, the second phase, the third phase, the fourthphase and the fifth phase in that order, performs the sixth phase andthe seventh phase in that order, performs the fifth phase, and theeighth phase and the ninth phase in that order, and also startsperforming the second phase and the sixth phase at the same time andperforming the third phase and the seventh phase at the same time.

[0246] Thus, an efficient pipeline process can be implemented whenjudging whether or not to perform the filter operation and performingthe filter operation, and so the acceleration of processing can beexpected. In addition, the pixel data is read in order of the firstvertical reference pixel string L_(YL), second vertical reference pixelstring L_(YR), second subject pixel string L_(SR) and second horizontalreference pixel string L_(XR) so that the reading of the pixel data ofthe first vertical reference pixel string L_(YL) and second verticalreference pixel string L_(YR) and the reading of the pixel data of thesecond subject pixel string L_(SR) and second horizontal reference pixelstring L_(XR) can be successively performed, allowing the readingprocess of the pixel data to be simplified.

[0247] Furthermore, according to this embodiment, the deblock filteringprocess calculates the values of the pixels after the deblock filteringv₀′ to v₂′ and v₅′ to v₇′ by the filter operation equations of the aboveequations (1) to (3) and (6) to (8) as to the pixel strings orthogonalto the boundary between mutually adjacent blocks.

[0248] It is thereby possible, when performing the filter operation asto one pixel, to further simplify the deblock filtering process becauseonly two pixel values are used instead of using an average of a largenumber of pixels. In addition, it is possible to implement the effectivefilter effect in addition to further simplifying the deblock filteringprocess because the computing is performed by the addition andsubtraction and bit shift (a division wherein a divisor is an integralmultiple of 2) using one pixel value and the value of the pixel of thesame pixel string which is the reference pixel in the adjacent block.

[0249] Furthermore, according to this embodiment, the deblock filteringprocess performs the filter operation as to one pixel of the pixels ofthe pixel string orthogonal to the boundary between mutually adjacentblocks when no mutual difference value of the pixels between that onepixel and the pixel of the same pixel string which is the referencepixel positioned on the boundary in the adjacent block is exceedingthresholds C₁ and C₂, where constant values are used as the thresholdsC₁ and C₂.

[0250] Thus, as the thresholds C₁ and C₂ are the constant values, it isnot necessary to calculate C₁ and C₂ in the course of the deblockfiltering process, and no filter operation is performed if any of themutual difference values of the pixels is exceeding the thresholds C₁and C₂. Therefore, it is possible to render the deblock filteringprocess relatively higher-speed compared to the past in addition tofurther simplifying it.

[0251] Furthermore, according to this embodiment, the threshold C₁ forcomparing the difference values between the reference pixels and theadjacent pixels adjacent thereto is set to be larger than the thresholdC₂ for comparing the difference values between the reference pixels andthe pixels other than the adjacent pixels.

[0252] Thus, a further effective filter effect can be implementedbecause the deblock filtering process having considered thecharacteristic that the average of luminance is different among theblocks can be performed.

[0253] Furthermore, according to this embodiment, the threshold C₁ forcomparing the difference values between the reference pixels and theadjacent pixels adjacent thereto is set at “32” assuming the maximumvalue of the pixels as “255,” and the threshold C₂ for comparing thedifference values between the reference pixels and the pixels other thanthe adjacent pixels is set at “16.”

[0254] Thus, a further effective filter effect can be implementedbecause the deblock filtering process having considered thecharacteristic that the average of luminance is different among theblocks can be performed.

[0255] Furthermore, according to this embodiment, the deblock filteringprocess judges whether or not to perform the filter operation as to thepixels in a direction sequentially going away from the boundary from theadjacent pixel adjacent to the reference pixel, and when it judges notto perform the filter operation as to a certain pixel, no filteroperation is performed as to that pixel and the pixels thereafter.

[0256] It is thereby possible to reduce the number of times of judgmentof whether or not to perform the filter operation as to the pixels so asto further accelerate the deblock filtering process.

[0257] Furthermore, according to this embodiment, the post filterportion 40 has the buffer capable of storing the pixel data equivalentto 21 pixels, and the deblock filtering process collectively reads allthe pixel data from that one pixel to the reference pixels from the VRAM35 to the buffer so as to perform the filter operation based on thepixel data of the buffer.

[0258] Thus, the deblock filtering process can be further acceleratedbecause, as to each pixel of the pixel string, the pixel data necessaryto perform the deblock filtering process is collectively read to thebuffer for batch processing.

[0259] Furthermore, according to this embodiment, the deblock filteringprocess performs the filter operation to one of the horizontal pixelstring or the vertical pixel string in the block, and then performs thefilter operation to the other.

[0260] Thus, a further effective filter effect can be implementedbecause the deblock filtering process is performed to both thehorizontal pixel string and the vertical pixel string in the block.

[0261] In the above embodiment, the motion picture data in the MPEGformat is corresponding to compressed image data according to claims 1,2, 9 and 12, and the VRAM 35 is corresponding to an image storing meansaccording to claim 3 or 5, and the buffer of the post filter portion 40is corresponding to the storing means for work according to claim 3, 5or 8. In addition, the reference pixels r_(x0) and r_(x1) arecorresponding to the horizontal reference pixels according to claim 2,4, 10 or 12, the reference pixels r_(y0) to r_(y7) are corresponding tothe vertical reference pixels according to claim 2, 3, 6, 10 or 12, andthe reference pixels r_(x0) and r_(x1) and the subject pixels v₀ to v₇are corresponding to the first pixel string according to claim 3 or 4.

[0262] In addition, according to the above embodiment, the referencepixels r_(y0) to r_(y7) are corresponding to the second pixel stringaccording to claim 3 or 5.

[0263] Moreover, while the above embodiment is constituted so that thehorizontal filter operation as to the subject pixels v₀ to v₇ isperformed when no mutual difference value of the pixels between that onepixel and the pixels of the same pixel string which are the referencepixels r_(x0) and r_(x1) positioned on the boundary in the adjacentblock is exceeding thresholds C₁ and C₂, it is not limited thereto andit may also be constituted so that, as with the vertical filteroperation, it determines whether or not to perform the horizontal filteroperation as to the subject pixels v₀ to v₇ based solely on the value ofthat one pixel and the values of the reference pixels r_(x0) and r_(x1).

[0264] In addition, in the above embodiment, all the cases of performingthe processes shown in the flowcharts in FIGS. 5 to 11 were described asto the cases of performing them by the hardware of the post filterportion 40. However, they are not limited thereto but these processesmay also be performed by the CPU 30, in which case it may be constitutedto execute the control program stored in the ROM 32 in advance or theprogram may be read to the RAM 34 from a storage medium storing theprogram showing procedures thereof.

[0265] Here, the storage media are semiconductor storage media such asthe RAM and ROM, magnetic memory type storage media such as FD and HD,optical reading method media such as CD, CDV, LD and DVD or the like,and magnetic memory type/optical reading method media such as MO,including any storage medium readable by the computer irrespective ofwhether the reading method is electronic, magnetic or optical.

[0266] In addition, according to the above embodiment, while the imageprocessing apparatus, image processing program and image processingmethod related to the present invention are applied to the cases ofperforming the deblock filtering process to the reproduced image whendecoding the motion picture data compressed in the MPEG format andreproducing the motion picture by the multitasking OS in the computer100 as shown in FIG. 1, they are not limited thereto but applicable toother cases as far as they do not deviate from the main point of thepresent invention. For instance, they are applicable to the cases ofdisplaying the images based on the image data compressed in the JPEGformat and the cases of expanding the image based on the compressedimage data compressed by the image compressing process for performingthe discrete cosine transform process and quantization process by apredetermined block.

[0267] As described above, the image processing apparatus according toclaim 1 of the present invention has the effect of simplifying thefiltering process for reducing the block noise when decoding the imagecompared to the past because it only needs to refer to the subjectpixels and the reference pixels when judging whether or not to performthe filter operation and performing the filter operation. In addition,in the case where the expanded image is stored in storing means such asa memory, it can reduce the number of times of access to the storingmeans so that, as its effect, the acceleration of processing can beexpected as to the image processing apparatus of which access to thestoring means is a critical path.

[0268] Moreover, the image processing apparatus according to claims 2 to8 of the present invention has the effect of simplifying the filteringprocess for reducing the block noise when decoding the image compared tothe past because it only needs to refer to the subject pixels,horizontal reference pixels and vertical reference pixels when judgingwhether or not to perform the filter operation and performing the filteroperation. In particular, it uses the pixels positioned on theboundaries in the horizontal reference block and vertical referenceblock as the horizontal reference pixels and vertical reference pixels,and so it has the effect of simplifying the filtering process withoutsignificantly reducing the filter effect. Furthermore, in the case wherethe expanded image is stored in the storing means such as the memory, itcan reduce the number of times of access to the storing means so that,as its effect, the acceleration of processing can be expected as to theimage processing apparatus of which access to the storing means is thecritical path.

[0269] Furthermore, the image processing apparatus according to claims 3to 8 of the present invention also has the effect of reducing the numberof times of reading to the image storing means because, when performingthe filter processing to the subject pixels of the first pixel string,it only has to read the pixels of the first and second pixel stringsfrom the image storing means.

[0270] Moreover, the image processing apparatus according to claims 4 to8 of the present invention also has the effect of reducing the number oftimes of reading to the image storing means because, when performing thefilter processing to n subject pixels, it only has to read 2n+1 pixelsfrom the image storing means. In addition, it has the effect ofimproving the filter effect because, when judging whether or not toperform the filter operation and performing the filter operation, thecomparatively adequate reference pixels are used according to the areawhich the subject pixels belong to.

[0271] Furthermore, the image processing apparatus according to claims 5to 8 of the present invention also has the effect of reducing the loadand implementing a comparatively simple configuration in case ofconstituting it by the hardware because the updating of the pointer forthe vertical reference pixel can be implemented just by the shifter andthe adder.

[0272] Moreover, the image processing apparatus according to claims 6 to8 of the present invention also has the effect of further reducing thenumber of times of reading to the image storing means because it is notnecessary to reread the pixel data of the first horizontal referencepixel string and first subject pixel string from the image storingmeans.

[0273] Furthermore, the image processing apparatus according to claim 8of the present invention can implement the efficient pipeline process atthe time of judging whether or not to perform the filter operation andperforming the filter operation so that, as its effect, the accelerationof processing can be expected. In addition, it reads the pixel data inorder of the first vertical reference pixel string, second verticalreference pixel string, second subject pixel string and secondhorizontal reference pixel string so that the reading of the pixel dataof the first vertical reference pixel string and second verticalreference pixel string and the reading of the pixel data of the secondsubject pixel string and second horizontal reference pixel string can besuccessively performed, and so it also has the effect of simplifying thereading process of the pixel data. Moreover, it also has the effect ofcomparatively reducing the circuit scale when constituting it by thehardware because it only requires work memory equivalent to 5n+1 pixelsas against the work memory equivalent to at least one frame used forwork in the past.

[0274] On the other hand, the image processing program according toclaim 9 of the present invention has the effect equivalent to that ofthe image processing apparatus according to claim 1.

[0275] Furthermore, the image processing program according to claim 10of the present invention has the effect equivalent to that of the imageprocessing apparatus according to claim 2.

[0276] On the other hand, the image processing method according to claim11 of the present invention has the effect equivalent to that of theimage processing apparatus according to claim 1.

[0277] Furthermore, the image processing method according to claim 12 ofthe present invention has the effect equivalent to that of the imageprocessing apparatus according to claim 2.

What is claimed is:
 1. An image processing apparatus for, based oncompressed image data compressed by an image compressing process forperforming a discrete cosine transform process and a quantizationprocess by a predetermined block, performing a filtering process to anexpanded image on or after expanding the image by said block, wherein,in the case where a block adjacent to a subject block is a referenceblock, and furthermore, one pixel in said subject block is a subjectpixel, and one pixel in said reference block is a reference pixel, saidfiltering process judges whether or not a difference value between avalue of said subject pixel and a value of said reference pixel isexceeding a threshold, determines whether or not to perform filteroperation as to said subject pixel based solely on said judgment result,and performs the filter operation as to said subject pixel based solelyon the value of said subject pixel and the value of said referencepixel.
 2. An image processing apparatus for, based on compressed imagedata compressed by an image compressing process for performing adiscrete cosine transform process and a quantization process by apredetermined block, performing a filtering process to an expanded imageon or after expanding the image by said block, wherein, in the casewhere a block horizontally adjacent to a subject block is a horizontalreference block, a block vertically adjacent to said subject block is avertical reference block, and furthermore, one pixel in said subjectblock is a subject pixel, and one pixel in said horizontal referenceblock located on a boundary with said subject block and at the samevertical position as said subject pixel is a horizontal reference pixel,and one pixel in said vertical reference block located on the boundarywith said subject block and at the same horizontal position as saidsubject pixel is a vertical reference pixel, said filtering processjudges whether or not a difference value between a value of said subjectpixel and a value of said vertical reference pixel is exceeding athreshold, said filtering process performs horizontal filter operationas to said subject pixel based solely on the value of said subject pixeland the value of said horizontal reference pixel, said filtering processdetermines whether or not to perform vertical filter operation as tosaid subject pixel based solely on said judgment result, and performsthe vertical filter operation as to said subject pixel based solely onthe value of said subject pixel and the value of said vertical referencepixel.
 3. The image processing apparatus according to claim 2, usablyconnecting to image storing means for storing said expanded image; andhaving storing means for work for storing pixel data for performing saidfiltering process, and wherein, in the case where a plurality of pixelsin said subject block and horizontally successive from the boundary withsaid horizontal reference block are said subject pixels, said filteringprocess collectively reads the pixel data of a first pixel stringcomprised of said plurality of subject pixels and said horizontalreference pixel and the pixel data of a second pixel string comprised ofvertical reference pixels corresponding to said respective subjectpixels into said storing means for work, makes said judgment as to eachof said subject pixels based on the pixel data of said storing means forwork, said filtering process performs horizontal filter operation as tosaid respective subject pixels based solely on the values of saidsubject pixels and the value of said horizontal reference pixel byreferring to the pixel data of said storing means for work, saidfiltering process determines whether or not to perform vertical filteroperation as to said respective subject pixels based solely on saidjudgment results corresponding to the subject pixels, and performs thevertical filter operation as to said respective subject pixels basedsolely on the values of said subject pixels and the values of saidvertical reference pixels by referring to the pixel data of said storingmeans for work.
 4. The image processing apparatus according to claim 3,constituting each of said blocks as a rectangular image area comprisedof 2n (n≧1) pieces of pixel horizontally and 2m (m≧1) pieces of pixelvertically, and dividing it into four areas of upper left, upper right,lower left and lower right comprised of n pieces of pixel horizontallyand m pieces of pixel vertically, and further constituting said firstpixel string with horizontal reference pixel and the pixels belonging tothe same area of said four areas; said filtering process tries saidfilter operation as to all the pixels of said expanded image whilerepeating reading of said pixel data, said judgment and said filteroperation as a series of processes; as to the subject pixel belonging tothe upper left area of said subject block, said filtering process usesthe values of the reference pixels of the horizontal reference block andthe vertical reference block adjacent to said subject block on the leftof and above it for said judgment and said filter operation; as to thesubject pixel belonging to the upper right area of said subject block,said filtering process uses the values of the reference pixels of thehorizontal reference block and the vertical reference block adjacent tosaid subject block on the right of and above it for said judgment andsaid filter operation; as to the subject pixel belonging to the lowerleft area of said subject block, said filtering process uses the valuesof the reference pixels of the horizontal reference block and thevertical reference block adjacent to said subject block on the left ofand below it for said judgment and said filter operation; and as to thesubject pixel belonging to the lower right area of said subject block,said filtering process uses the values of the reference pixels of thehorizontal reference block and the vertical reference block adjacent tosaid subject block on the right of and below it for said judgment andsaid filter operation.
 5. The image processing apparatus according toclaim 4, wherein said image storing means stores the pixel data of thepixels constituting said expanded image in order of left to right andtop to bottom, said filtering process refers to a pointer for thevertical reference pixel pointing at a storage address of the pixel dataof said second pixel string in said image storing means and reads thepixel data of said second pixel string into said storing means for work,and furthermore, in the case where the number of horizontal pixels ofsaid expanded image is X and a vertical coordinate of the subject pixelin said subject block is y, as to each of the above described series ofprocesses, said filtering process adds a value equivalent to the numberof the subject pixels to be collectively processed in the process to thevalue of said pointer for the vertical reference pixel, when the subjectblock to be processed next is a leftmost block in said expanded imageand a remainder on dividing y by 2m is “0”, said filtering processsubtracts 2X from the value of said pointer for the vertical referencepixel, when the subject block to be processed next is the leftmost blockin said expanded image and the remainder on dividing y by 2m is m, saidfiltering process adds 2mX to the value of said pointer for the verticalreference pixel, when the subject block to be processed next is theleftmost block in said expanded image and the remainder on dividing y by2m is a value other than “0” or m, said filtering process and subtractsX from the value of said pointer for the vertical reference pixel. 6.The image processing apparatus according to claim 5, wherein, in thecase where the pixel string comprised of a left half of the pixels onthe same horizontal line in said subject block is a first subject pixelstring, the pixel string comprised of a right half thereof is a secondsubject pixel string, and furthermore, the pixel string in saidhorizontal reference block and adjacent to said first subject pixelstring on the left of it and comprised of the same number of pixels asthat of said second subject pixel string is a first horizontal referencepixel string, and the pixel string in said horizontal reference blockand adjacent to said second subject pixel string on the right of it andcomprised of the same number of pixels as that of said first subjectpixel string is a second horizontal reference pixel string, andfurthermore, the pixel string comprised of the vertical reference pixelscorresponding to the subject pixels of said first subject pixel stringis a first vertical reference pixel string, and the pixel stringcomprised of the vertical reference pixels corresponding to the subjectpixels of said second subject pixel string is a second verticalreference pixel string, said filtering process reads the pixel data ofsaid second subject pixel string, said second horizontal reference pixelstring, said first vertical reference pixel string and said secondvertical reference pixel string into said storing means for work as toeach of said series of processes, also reuses the pixel data of saidsecond subject pixel string and said second horizontal reference pixelstring read in an immediately preceding process as the pixel data ofsaid first horizontal reference pixel string and said first subjectpixel string, and performs said judgment and said filter operation as toall the subject pixels of said first subject pixel string and saidsecond subject pixel string.
 7. The image processing apparatus accordingto claim 6, wherein said series of processes performed first for saidexpanded image reads the pixel data of said first subject pixel string,said second subject pixel string, said second horizontal reference pixelstring, said first vertical reference pixel string and said secondvertical reference pixel string into said storing means for work, andsaid series of processes performed for the second time and thereafterreads the pixel data of said second subject pixel string, said secondhorizontal reference pixel string, said first vertical reference pixelstring and said second vertical reference pixel string into said storingmeans for work, and also reuses the pixel data of said second subjectpixel string and said second horizontal reference pixel string read inan immediately preceding process as the pixel data of said firsthorizontal reference pixel string and said first subject pixel string.8. The image processing apparatus according to claim 6 or 7, wherein, inthe case where the process of copying the pixel data of said secondsubject pixel string and said second horizontal reference pixel stringread in an immediately preceding process as the pixel data of said firsthorizontal reference pixel string and said first subject pixel stringrespectively is the first phase, the process of reading the pixel dataof said first vertical reference pixel string into said storing meansfor work is the second phase, the process of reading the pixel data ofsaid second vertical reference pixel string into said storing means forwork is the third phase, the process of reading the pixel data of saidsecond subject pixel string into said storing means for work is thefourth phase, the process of reading the pixel data of said secondhorizontal reference pixel string into said storing means for work isthe fifth phase, the process of performing said horizontal filteroperation as to each subject pixel of said first subject pixel stringbased on the pixel data of said first horizontal reference pixel stringis the sixth phase, the process of performing said judgment and saidvertical filter operation as to each subject pixel of said first subjectpixel string based on processing results of said sixth phase and thepixel data of said first vertical reference pixel string is the seventhphase, the process of performing said horizontal filter operation as toeach subject pixel of said second subject pixel string based on thepixel data of said second horizontal reference pixel string is theeighth phase, and the process of performing said judgment and saidvertical filter operation as to each subject pixel of said secondsubject pixel string based on the processing results of said eighthphase and the pixel data of said second vertical reference pixel stringis the ninth phase, said series of processes performs said first phase,said second phase, said third phase, said fourth phase and said fifthphase in that order, perform said sixth phase and said seventh phase inthat order, perform said fifth phase, and said eighth phase and saidninth phase in that order, and also start performing said second phaseand said sixth phase at the same time and performing said third phaseand said seventh phase at the same time.
 9. An image processing programfor, based on compressed image data compressed by an image compressingprocess for performing a discrete cosine transform process and aquantization process by a predetermined block, causing a computer toperform a filtering process to an expanded image on or after expandingan image by said block, wherein, in the case where a block adjacent to asubject block is a reference block, and furthermore, one pixel in saidsubject block is a subject pixel, and one pixel in said reference blockis a reference pixel, said filtering process judges whether or not adifference value between a value of said subject pixel and a value ofsaid reference pixel is exceeding a threshold, determines whether or notto perform filter operation as to said subject pixel based solely onsaid judgment result, and performs the filter operation as to saidsubject pixel based solely on the value of said subject pixel and thevalue of said reference pixel.
 10. An image processing program for,based on compressed image data compressed by an image compressingprocess for performing a discrete cosine transform process and aquantization process by a predetermined block, causing a computer toperform a filtering process to an expanded image on or after expandingthe image by said block, wherein, in the case where a block horizontallyadjacent to a subject block is a horizontal reference block, a blockvertically adjacent to said subject block is a vertical reference block,and furthermore, one pixel in said subject block is a subject pixel, andone pixel in said horizontal reference block located on a boundary withsaid subject block and at the same vertical position as said subjectpixel is a horizontal reference pixel, and one pixel in said verticalreference block located on the boundary with said subject block and atthe same horizontal position as said subject pixel is a verticalreference pixel, said filtering process judges whether or not adifference value between a value of said subject pixel and a value ofsaid vertical reference pixel is exceeding a threshold, said filteringprocess performs horizontal filter operation as to said subject pixelbased solely on the value of said subject pixel and the value of saidhorizontal reference pixel, said filtering process determines whether ornot to perform vertical filter operation as to said subject pixel basedsolely on said judgment result, and performs the vertical filteroperation as to said subject pixel based solely on the value of saidsubject pixel and the value of said vertical reference pixel.
 11. Animage processing method for, based on compressed image data compressedby an image compressing process for performing a discrete cosinetransform process and a quantization process by a predetermined block,performing a filtering process to an expanded image on or afterexpanding the image by said block, wherein, in the case where a blockadjacent to a subject block is a reference block, and furthermore, onepixel in said subject block is a subject pixel, and one pixel in saidreference block is a reference pixel, said filtering process judgeswhether or not a difference value between a value of said subject pixeland a value of said reference pixel is exceeding a threshold, determineswhether or not to perform filter operation as to said subject pixelbased solely on said judgment result, and performs the filter operationas to said subject pixel based solely on the value of said subject pixeland the value of said reference pixel.
 12. An image processing methodfor, based on compressed image data compressed by an image compressingprocess for performing a discrete cosine transform process and aquantization process by a predetermined block, performing a filteringprocess to an expanded image on or after expanding the image by saidblock, wherein, in the case where a block horizontally adjacent to asubject block is a horizontal reference block, a block verticallyadjacent to said subject block is a vertical reference block, andfurthermore, one pixel in said subject block is a subject pixel, and onepixel in said horizontal reference block located on a boundary with saidsubject block and at the same vertical position as said subject pixel isa horizontal reference pixel, and one pixel in said vertical referenceblock located on the boundary with said subject block and at the samehorizontal position as said subject pixel is a vertical reference pixel,said filtering process judges whether or not a difference value betweena value of said subject pixel and a value of said vertical referencepixel is exceeding a threshold, said filtering process performshorizontal filter operation as to said subject pixel based solely on thevalue of said subject pixel and the value of said horizontal referencepixel, said filtering process determines whether or not to performvertical filter operation as to said subject pixel based solely on saidjudgment result, and performs the vertical filter operation as to saidsubject pixel based solely on the value of said subject pixel and thevalue of said vertical reference pixel.